When planning Census 2021, we recognised that the census results would only be of value when they were fit for purpose and trusted by users as a basis for decisions. High-quality statistics rely on having good-quality processes all the way from designing the questions to preparing the final estimates for publication.
This report summarises the quality assurance we conducted on the census data to check that:
the data were being processed correctly
unexpected features of the data were dealt with appropriately
the final estimates were plausible in the context of the other information available to us
This adds to other published information on the census quality assurance.Nôl i'r tabl cynnwys
Our planning of the census quality assurance (QA) activities started in 2018. We began by outlining what that work needed to cover, and then we started work specifying and acquiring the other data sources we intended to use in our analysis. We used the 2019 Census Rehearsal as a test of our first methods and tools and, in January 2020, we published our planned strategy and approach to census QA.
The published planned approach evolved as a result of users' feedback, developments in availability of comparator data, and the priorities that emerged from analysis of the census data. Major changes to the published planned approach are highlighted in this report.Nôl i'r tabl cynnwys
This strand of our quality assurance looked at how the census processes worked, from respondents completing the questionnaire to how we imputed data for people missed from census responses. It also monitored the census data moving through different stages of processing.
This strand went largely as planned. As in 2011, we ran a Census Quality Survey to assess how accurately individual respondents completed each question, and we will publish the results of this survey. Initial quality assurance for other processes was part of the standard work for the team responsible for that process. Analytical teams conducted further assurance as the data progressed through each stage of processing.
This two-stage approach to the assurance of processes allowed us to be flexible in where we concentrated our efforts. It allowed us to ensure all processes were checked but prioritise areas where processes had not worked as planned or where the data contained unexpected features that required us to make changes to our statistical design.
Some examples of issues identified through this assurance of processes are:
data capture - the original coding rules for paper questionnaires did not reflect the different ordering of the tick boxes for the national identity question on the Welsh version of the questionnaire; when we identified this issue, we recoded the records affected to correct the data
addressing - we used an address matching tool to identify the addresses provided as responses to the questions on address one year ago, term-time address and visitor address; we found that when the address provided was in Northern Ireland, the tool sometimes matched this to an address in England and Wales, so we corrected this process to ensure that these addresses were correctly assigned to Northern Ireland
misclassification - we received a small number of returns where the respondent had completed the wrong type of questionnaire, which meant that some household addresses were being classified as communal establishments; we identified and corrected them using other information to validate the change of address type
This strand of the work looked at whether the statistics based on the census data were plausible in the context of other information available to us.
The national population estimates derived from the census were assessed by a team of demographic analysts. This assessment covered, among - the size and age-structure of the population
sex ratios at each age
population distribution among households of different sizes and different types of communal establishment
evaluation of implied fertility, mortality, and migration since the 2011 Census
Population estimates for each of the 331 local authorities in England and Wales went through a comprehensive set of standard checks. We also conducted further investigation of any evidence provided by a local authority as part of our Local Authority Insight initiative described in the following section. A "case study" using a fictional area, illustrating the checks we conducted and how the evidence from different sources was evaluated, is provided in our Quality assuring the local authority census population estimates, England and Wales methodology article.
In addition to our assurance of the population estimates, we also conducted detailed assurance of the census results for each topic included on the census questionnaire. This analysis included:
assessments of response-rate profiles
comparisons with available data sources on each topic
comparisons with 2011 subnational patterns
consideration of any potential issues raised by users or arising elsewhere in the quality assurance process
We will be publishing more information on the findings of our quality assurance work for each topic alongside the results for that topic.Nôl i'r tabl cynnwys
The work described in the previous sections generated a huge amount of evidence on the quality of the census data. However, this was only of value when we could assess it and act on any issues we identified. Our approach to this was consistent with that described in our published planned strategy and approach.
The first part of this aspect of the work was the daily Data Quality Management Forum (DQMF) meetings. These brought together representatives of the teams processing or assuring the data, allowing a very quick identification of issues together with an initial proposal of how they should be addressed.
Issues and proposed actions from the DQMF were considered at operational management meetings to endorse the proposed approach or agree an alternative, and to make sure that the necessary actions were integrated into the overall plan. Where necessary, the issue was discussed in more detail at a Statistical Contingencies Escalation Forum.
We used quality assurance (QA) panels, consisting of two or more Office for National Statistics (ONS) experts on population statistics who were independent of the QA team, to evaluate the evidence we collated on population estimates (nationally and for each local authority) and the topic results. We ran more than 100 panels looking at population estimates, making sure that each area received individual consideration both in the initial investigation and in the evaluation of that investigation. The vast majority of panels resulted in the estimates presented being endorsed as of good quality and suitable for publication. Where a panel did not endorse the estimates, these were escalated for further discussion.
Where the conclusion of the panel's work was that there was a quality issue that needed to be addressed, there were a number of possible actions. These included:
direct editing of records where a process had not worked as desired
adjustment of the statistical models through which estimates are produced
the addition of new methods
the development of quality notes to accompany statistics alongside their release
More information on adjustments resulting from this work is provided in Maximising the quality of Census 2021 population estimates.Nôl i'r tabl cynnwys
Office for National Statistics (ONS), released 7 November 2022, ONS website, methodology, How we assured the quality of Census 2021 estimates
Manylion cyswllt ar gyfer y Methodoleg
Ffôn: +44 1329 444972