Quality assurance of administrative data used in estimates of household final consumption expenditure as published in Consumer trends
This publication documents the quality assurance (QA) assessment of all data sources used to produce estimates of household final consumption expenditure published in Consumer trends.
Household final consumption expenditure (HHFCE) is spending by households on the goods and services they consume. It does not include investments (such as buying a house or valuables). Transactions between households are excluded because there is no overall impact on how much households consume. It is produced on a quarterly basis and contributes over 60% to the expenditure measure of gross domestic product (GDP). Provisional high-level estimates form part of the GDP first release. More detailed figures are published in the Quarterly National Accounts and Consumer trends three months after the end of the latest quarter. The statistics are also supplied to Eurostat, the statistical office of the European Union. HHFCE is a National Statistic.
Estimates of HHFCE are compiled from over 30 data sources. Some are produced internally by the Office for National Statistics (ONS), such as survey data from the Living Costs and Food Survey (LCF), and others are supplied by external bodies from the public and private sectors.
The UK Statistics Authority Code of Practice requires producers of official statistics to review the quality of their statistics. To improve our understanding of the quality of data sources used to estimate HHFCE, we conducted an exercise to review the administrative data used to share with our users.
In summer 2017, we sent detailed questionnaires to all of our data suppliers (35 in total) and the results were assessed using the Quality Assurance of Administrative Data (QAAD) toolkit.
A level of assurance was assigned to each of the suppliers from “basic” to “enhanced” to “comprehensive”. Of the 35 questionnaires dispatched, 17 were assigned as comprehensive, six as enhanced, seven as basic and five as non-responders. The non-responders were contacted at least five times by letter and by telephone.
The overall quality of data sources used to compile HHFCE was assessed to be of low-quality concern and medium public interest which equates to an overall risk profile of A1/A2.Nôl i'r tabl cynnwys
Under the Code of Practice, statistics producers must review the quality of data used to produce official statistics. The UK Statistics Authority encourages the use of the Quality Assurance of Administrative Data (QAAD) toolkit.
More information about the UK Statistics Authority QAAD initiative can be found on the website of its regulatory arm, the Office for Statistics Regulation.
We have used the toolkit to investigate the quality of data used to compile estimates of spending on goods and services by members of UK households. This exercise covers both administrative and other data sources, for example survey sources, in line with the guidelines provided by the UK Statistics Authority. We sent out questionnaires to all the external and internal data suppliers to collect information on their processes of quality assurance.Nôl i'r tabl cynnwys
Use of the Quality Assurance of Administrative Data (QAAD) toolkit
We used the Quality Assurance of Administrative Data (QAAD) toolkit to determine compliance for each of the 35 household final consumption expenditure (HHFCE) data sources. Although the QAAD is designed for administrative data many of the principles contained within it also apply to other data sources, so we used it to assess the quality of data provided by all the HHFCE suppliers.
The QAAD toolkit is built around the Quality Assurance (QA) Matrix which presents the levels of assurance for four areas of practice related to the quality assurance of official statistics and of the administrative data used to produce them. These are:
operational context and administrative data collection
communication with data supply partners
quality assurance principles, standards and checks applied by data suppliers
producer quality assurance investigations and documentation
Table 1 shows the four possible outcomes from the assessment.
|A0: No assurance||Not compliant with the Code of Practice for Official Statistics|
|A1: Basic assurance||Statistical producer has reviewed and published a summary of the administrative data QA arrangements|
|A2: Enhanced assurance||Statistical producer has evaluated the administrative data QA arrangements and published a fuller description of the assurance|
|A3: Comprehensive assurance||Statistical producer has investigated the administrative data QA arrangements, identified the results of independent audit, and published detailed documentation about the assurance and audit|
Download this table Table 1: Assurance levels used to assess each data source.xls .csv
The QAAD toolkit recognises that a proportionate approach is appropriate, whereby some statistics will need greater levels of assurance than others. The levels can be determined by an evaluation of:
the likelihood of quality concerns arising in the administrative data that may affect the quality of the statistics
the nature of the public interest served by the statistics
These two levels are used to derive the overall risk of data quality concerns using the Risk Profile Matrix (Table 2).
|Level of risk of quality concerns||Public interest profile|
|Low||Statistics of lower quality concern and lower public interest|
|Statistics of low quality concern and medium public interest|
|Statistics of low quality concern and higher public interest |
|Medium||Statistics of medium quality concern and lower public interest |
|Statistics of medium quality concern and medium public interest|
|Statistics of medium quality concern and higher public interest |
|High||Statistics of higher quality concern and lower public interest |
|Statistics of higher quality concern and medium public interest|
|Statistics of higher quality concern and higher public interest |
Download this table Table 2: UK Statistics Authority Risk and Profile Matrix.xls .csv
Use of the toolkit to assess the quality of household final consumption expenditure (HHFCE) data sources
Our approach to assessing the quality of the household final consumption expenditure (HHFCE) data sources involved the following steps:
We sent a questionnaire to each of the external and internal Office for National Statistics (ONS) data suppliers. This questionnaire requested information on their data collection activities, data processing activities and how their statistics are used. See Annex A for an example of the questionnaire.
We contacted non-responders to the survey by telephone, email or letter. 30 completed questionnaires were returned out of a total of 35 data suppliers.
We reviewed each of the responses received from the data suppliers to assess the strength of the quality assurance processes and the data processing operations. We did this by assessing the information they provided against eight categories:
- processes used to produce the statistics
- information about data sources
- revisions policy
- use of the data
- strengths and weaknesses of the data
- quality assurance checks
- quality assurance reviews
In our assessment of the 35 data sources, we marked each of their responses to the individual questions with the rating of either “no assurance”, “basic”, “enhanced” or “comprehensive”. We used this information to determine a level of risk of data quality concerns for each data source. Two people assessed each of the 35 data sources independently before reconciling their results.
The importance of data quality for different items of expenditure depends on their contribution to overall household spending. We allocated weights to each data source according to the contribution to the overall level of household final consumption expenditure in the UK by resident and non-resident households. We used this as our measure of public interest in the official statistics produced from the data source.
We used the level of risk of data quality concerns, and public interest profile allocated to each of the 35 data suppliers to produce a rating of risk of data quality concerns and public interest profile for the overall HHFCE statistical output.
The household final consumption expenditure (HHFCE) statistics are compiled using a total of 35 data sources (Table 3).
|Data source||COICOP (1999) classification|
|Living Costs and Food Survey (LCF)¹||01 (food and non-alcoholic beverages), 03 (clothing and footwear), 04 (housing water, gas and electricity), 05 (furnishings, household equipment), 06 (health), 07 (transport), 08 (communication), 09 (recreation and culture), 11 (restaurants and hotels), 12 (miscellaneous goods and services)|
|Ministry of Housing, Communities and Local Government (MHCLG)||04 (housing water, gas and electricity)|
|Valuation Office Agency (VOA)||04 (housing water, gas and electricity)|
|Annual Business Survey (ABS)¹||03 (clothing and footwear), 04 (housing, water, gas and electricity), 05 (furnishings, household equipment), 06 (health), 07 (transport), 08 (communication), 09 (recreation and culture), 12 (miscellaneous goods and services)|
|Department for Business, Energy and Industrial Strategy (BEIS)||04 (housing water, gas and electricity), 07 (transport)|
|Retail Sales Index (RSI)¹||03 (clothing and footwear), 04 (housing water, gas and electricity), 05 (furnishings, household equipment), 06 (health), 08 (communication), 09 (recreation and culture), 12 (miscellaneous goods and services)|
|Financial Corporations – Monetary Financial Institutions (FISIM (Financial Intermediation Services Indirectly Measured))¹||12 (miscellaneous goods and services)|
|HMRC (Her Majesty’s Revenue and Customs) – alcohol||02 (alcoholic beverages, tobacco and narcotics)|
|International Passenger Survey (IPS)¹||07 (transport)|
|HMRC – tobacco||02 (alcoholic beverages, tobacco and narcotics)|
|CGA||11 (restaurants and hotels)|
|A C Nielsen||02 (alcoholic beverages, tobacco and narcotics)|
|Department for Transport (DfT) – car data||07 (transport)|
|UK Trade and Transfers¹||11 (restaurants and hotels), TOUREX (foreign tourist expenditure), TOURIM (UK tourist expenditure abroad)|
|Consumer Price Index including owner occupiers’ housing costs (CPIH)¹||Used to deflate (remove the effects of price inflation from) a large number of 4 digit COICOPs|
|BSKYB||09 (recreation and culture)|
|Office of Water Services (OFWAT)||04 (housing water, gas and electricity)|
|Financial corporations – insurance, pensions¹||12 (miscellaneous goods and services)|
|Office of Communications (OFCOM)||08 (communication)|
|DfT – bus deflator||07 (transport)|
|HMRC – gambling||09 (recreation and culture)|
|Population and Public Policy¹||12 (miscellaneous goods and services)|
|ONS Vital Statistics¹||12 (miscellaneous goods and services)|
|Association of British Insurers||12 (miscellaneous goods and services)|
|DfT – sea||07 (transport)|
|Crime Survey for England and Wales (CSEW)||02 (alcoholic beverages, tobacco and narcotics)|
|Office of Rail and Road (ORR)||07 (transport)|
|Camelot||09 (recreation and culture)|
|Gambling Commission||09 (recreation and culture)|
|Transport for London (TfL)||07 (transport)|
|Gross fixed capital formation (GFCF)¹||07 (transport)|
|Civil Aviation Authority (CAA)||07 (transport)|
|Scottish Water||04 (housing water, gas and electricity)|
|Society of Motor Manufacturers and Traders (SMMT)||07 (transport)|
Download this table Table 3: List of data sources used in the compilation of household final consumption expenditure (HHFCE).xls .csv
Each of the completed questionnaires received from the data suppliers was reviewed to ascertain the level of quality assurance applied to the data, and whether the submitted evidence complies with the Code of Practice.
Eleven of the data sources are internal, that is, surveys or outputs which are operated by the Office for National Statistics (ONS). Annex B at the end of this document lists the ONS publications for these data sources. Twenty-four of the data sources are external and these range from private research companies producing consumer product prices to government departments and other public bodies. Five of the 35 data suppliers were non-responders to our survey and these were all external suppliers. The combined weight of these non-responders, in terms of their contribution to the overall level of household final consumption expenditure (HHFCE), is not judged to be significant (5%).
A weight was allocated to each data source according to the contribution to the overall level of HHFCE domestic data (Table 4). There is a high overall weighting for internal sources when compared to external sources.
|Data source||Overall weight||Overall assurance level|
|5 non-responding data sources||5%||No assurance|
Download this table Table 4: Weights of internal and external household final consumption expenditure (HHFCE) data sources.xls .csv
Overall assessment of data sources
The overall risk of data quality concerns was low, with 23 out of 30 data sources rated as having enhanced or comprehensive assurance levels (Table 5).
|Data source||Overall assurance level|
Download this table Table 5: Overall assurance level for household final consumption expenditure (HHFCE) data sources.xls .csv
The public interest profiles for the data sources based on their weights were medium or low, giving an overall profile of medium for total HHFCE.
This gives an overall risk profile of A1/A2 – statistics of low quality concern and medium public interest.
Analysis of internal sources
Table 6 below shows our assessment of the different criteria of the Quality Assurance Matrix for the internal data suppliers; it also contains our comments on the responses provided for each area of processing.
|Quality assurance categories||Assessment of the quality assurance processes||Comments|
|Information about data sources||8||3||0||0||0||The majority of the data suppliers provide evidence of very detailed information about their sources which is readily available on the ONS website.|
|Quality assurance checks||6||4||1||0||0||The majority of the data suppliers demonstrated that the data which they process is subject to thorough quality checks and were shown to have detailed documentation available about their quality assurance processes.|
|Quality assurance reviews||7||2||2||0||0||A large number of data suppliers have recently carried out formal quality reviews on their quality assurance processes.|
|Communication||9||1||0||1||0||There was evidence of very good methods of regular communication with data suppliers; many suppliers hold regular face to face meetings with suppliers to discuss specific issues. The data source with no assurance has a weight of less than 0.5%.|
|Processes used to produce the statistics||8||2||1||0||0||The main ONS survey sources have provided very detailed descriptions for activities such as sampling methods and questionnaire dispatch, editing and validation, disclosure controls and methods of deflation and indexing; much of this information is easily accessible on the ONS survey’s specific page on the ONS website.|
|Use of the data||9||2||0||0||0||The vast majority of the internal suppliers were able to give us a detailed description of the different users of their data.|
|Strengths and weaknesses of the data||4||4||1||0||2||The internal data suppliers provided us with a high standard of information on the strengths and the weaknesses of their data. Several said they were the chief source of information on this matter.|
|Revisions policy||11||0||0||0||0||The vast majority of suppliers have formal revisions policies in place.|
Download this table Table 6: Analysis of internal sources.xls .csv
Analysis of external sources
Table 7 below shows our assessment of the different criteria of the Quality Assurance Matrix for the external data suppliers; it also contains our comments on the responses provided for each area of processing. It should be noted that some of the quality assurance categories are not applicable for some of the data suppliers.
|Quality assurance categories||Assessment of the quality assurance processes||Comments|
|Information about data sources||10||5||4||0||5||The predominant response received was indicative of a high standard throughout our sample. Of the 24 external data suppliers, 15 of these are government departments or agencies which provide descriptions of their data sources in great detail on their respective websites.|
|Quality assurance checks||3||9||7||0||5||Many of the respondents carry out thorough and systematic checks of their data such as comparisons with similar data sources and looking at historical trends to identify anomalies. Also, quality assurance checks are carried out by heads of profession before the data is sent.|
|Quality assurance reviews||0||10||8||1||5||Many of the government departments review their quality assurance arrangements annually. However, there is weakness in this area as a large number of the private suppliers do not have arrangements in place to carry out regular reviews.|
|Communication||11||4||4||0||5||The majority of suppliers have very good systems of communication in place. Most suppliers reported that they had good, close working relationships with their suppliers such as regular email correspondence, telephone conversations and face to face meetings if specific issues arose.|
|Processes used to produce the statistics||11||7||1||0||5||Most processes are explained very well. The ten suppliers which were rated comprehensive were government departments or other public bodies; they had very detailed documentation about data processes which was readily available on their websites.|
|Use of the data||9||4||6||0||5||Many of the suppliers provided us with detailed information about users and highlighted background documents which are stored on their websites.|
|Strengths and weaknesses of the data||6||8||4||0||6||The government departments again provided a high level of detailed information about the advantages and disadvantages of their data, with many of them highlighting the benefit of being accredited as a National Statistic.|
|Revisions policy||10||2||3||1||8||The overriding majority of the respondents marked as comprehensive were government departments which document their detailed revisions policies on their websites; these policies are in line with the Code of Practice for Statistics.|
Download this table Table 7: Analysis of external sources.xls .csv
Quality assurance checks carried out by the Office for National Statistics (ONS)
The household final consumption expenditure (HHFCE) branch of the ONS carries out extensive quality assurance checks on the data which it produces. The checks carried out by the team are as follows:
clear desk instructions are in place for quality assurance processes and an issues log maintained
data files received from each data source are checked to ensure they appear in line with past deliveries; suppliers are contacted to confirm any anomalies or, if necessary, to request resupply of the data
revisions to the HHFCE data are checked by analysts; briefings sent by data suppliers are used to validate the revisions
following the results run consistency checks are carried out to ensure accuracy
growth rates are reviewed using graphs; estimates are corroborated against other survey data to assess consistency
internal briefings are produced which record our data during the quarterly balancing process
the finalised HHFCE data is checked by all team members prior to publication
Adjustments to data as part of supply and use (SU) balancing
After source data has been processed preliminary estimates of HHFCE by product are compared with estimates derived from other sources, including business surveys, as part of supply and use (SU) balancing.
Gross domestic product (GDP) in current prices can be measured using three different approaches: production, income and expenditure. In the UK, we compile and draw on information from all three approaches and balance them together to arrive at a single estimate of GDP which uses all available data.
Following international guidance and best practice, we confront the estimates from the three approaches using the supply and use tables (SUTs) framework. This reconciles, at a detailed level, the supply of goods and services produced domestically and imported with their use – either consumed, invested or exported.
The HHFCE estimates form part of the expenditure measure of GDP and appear in the use table. They are therefore included in the annual balancing process. The balancing process takes place at the 112 products level (CPA 2008). Adjustments to HHFCE data from the SU balancing process are then allocated to the unbalanced HHFCE estimates at the COICOP (Classification of individual consumption by purpose) level using a CPA (Classification of product by activity) to COICOP mapping.
Strengths and weaknesses of estimates of household final consumption expenditure (HHFCE) by individual classification of individual consumption by purpose (COICOP)
The HHFCE estimates are produced according to well-established and well-documented methods and comply with international standards such as the System of National Accounts: SNA 08 and European System of Accounts: ESA 2010.
The estimates are compiled by experienced analysts using integrated data processing systems and employing extensive data checking techniques.
The supply and use balancing process reconciles estimates of household spending on specific products with estimates of the value of output produced by businesses. Respondents to household surveys may over or underestimate expenditure on some products; in these cases information from business surveys is used to adjust the estimate of household expenditure.
Patterns of household expenditure keep changing so it is important to review methods used to estimate HHFCE regularly to ensure all relevant expenditure is captured. We have an ongoing programme to review the methodology used for HHFCE.
HHFCE is currently produced using COICOP 1999 – patterns of consumer spending have changed considerably since then. For example, COICOP 1999 treats postal services as communication (COICOP 8) whereas now the expansion in online shopping means most of this is delivery of parcels so it sits better within Transport services (COICOP 7). A new expanded version – COICOP 2018 (2.45MB) – has just been produced and we are considering when to adopt this for HHFCE statistics.
The current systems used to produce HHFCE are complex. They have recently been improved but it takes a long time to make changes to the methods used to calculate HHFCE. Further investment would enable estimates of HHFCE to reflect changes in consumer spending more quickly.
We will continue to work closely with both internal and external suppliers to improve the quality of data used to produce household final consumption expenditure (HHFCE). Improved quality documentation and procedures will be requested in future service level agreements and contracts.
We will ensure data quality is discussed at all regular meetings with suppliers including the importance of holding regular quality assurance reviews.
We will seek updates about any methodology or policy changes that could affect the data.
We will seek further clarification about data quality from sources rated as basic or having no assurance.
When engaging in the future with our external data suppliers, we will highlight the importance of formal revisions policies which again will be fully documented in future service level agreements and contracts.
The review of our quality assurance of administrative data will now become part of our ongoing quality review procedures.
Manylion cyswllt ar gyfer y Methodoleg
Ffôn: +44 (0)1633456658