Criteria for moving to admin-based population estimates as official estimates of population

This article sets out the criteria for acceptance of the admin-based population estimates (ABPEs) as the official estimates of the population.

The ABPEs are official statistics in development, currently published alongside our traditional-methods mid-year estimates (MYEs). We plan for the ABPEs to become the official population estimates for England and Wales in 2025, subject to an assessment against the criteria presented in this article. We do not necessarily expect to meet every criterion in full before moving to the ABPEs as the official estimates of the population. Instead, assessment against the suite of criteria will guide our decision on whether we are ready to move to the new method.

Population statistics have a wide range of uses, including for national and local service planning. The decision to move to the ABPEs as our official estimates needs to be evidence based and transparent. Therefore, we have produced a set of criteria that will inform this decision. These criteria are set in advance, so that there is an opportunity for feedback and clarity on the basis for the decision.

Additionally, we requested an assessment of the ABPEs from the Office for Statistics Regulation (OSR), as part of our plans for the ABPEs to become the official population estimates. The OSR published phase one of its review of the ABPEs in July 2024. The review included the requirement that we define and publish the criteria to support our decision for the ABPEs to become the official population estimates. In response, we described our plans to publish our proposed acceptance criteria in our Action plan in response to OSR assessment of Office for National Statistics (ONS) ABPE methodology on 28 October 2024. This article presents these criteria.

What the criteria mean

The ABPEs will become the official estimates when we are confident the criteria presented in this article can be met to an acceptable standard. This means the ABPEs will be used by us and endorsed as the official estimates of the population for England and Wales. The accompanying advice will be for the ABPEs to be the estimates for users when considering all topics that require a measure of the population. These might be policy or budgetary decisions in local and central government departments, or for survey weighting.

We aim to meet the standards of official statistics set out in the Code of Practice for Statistics. However, this is not the same as becoming accredited official statistics, which is determined following an assessment by the OSR. We are working to meet the standards expected of accredited official statistics by summer 2025.

Approach to the criteria

We have developed a set of criteria to inform our decision on whether we will move to the APBEs as the official estimates of the population. The criteria have been compiled in the context of the OSR's review of the ABPEs and our subsequent action plan. Where appropriate, the criteria are aligned with the European Statistical System Quality Dimensions, which provides a definitive framework to assess quality.

User needs have also been a key consideration in the development of these criteria. Conversations with a range of users have indicated that user needs include certainty around the reliability of administrative data supply, data quality, and harmonisation and coherence across sources and different parts of the UK. These needs are reflected in the criteria.

When presenting the criteria in this article, we give context and background and report on our progress to date, where appropriate. Examples of the evidence we will use to assess each criterion are also provided. We will regularly monitor progress towards these criteria as our methods develop. The findings of this exercise will be set out in a further publication by summer 2025. This will give our assessment of the ABPEs against the criteria before the mid-2024 population estimates are published in summer 2025.

User engagement on the criteria

Moving to the ABPEs as the official population estimates is an important step towards a population and migration statistical system that makes greater use of administrative data. It is important for a wide range of users, including users from central government departments, devolved governments and local authorities, academics, and businesses.

The criteria have been discussed and tested with users, including at expert meetings and focus groups. These engagement exercises have provided a valuable assessment of what is important and what we should consider when moving to the ABPEs as the official population estimates. We have considered this user feedback in the criteria presented in this article.

We intend to keep users updated on our progress via our Quarterly update on population and migration statistics articles. We will also publish our assessment of the ABPEs against the criteria before the mid-2024 population estimates are published in summer 2025. We continue to welcome feedback on this approach via pop.info@ons.gov.uk.

We have defined 17 criteria to consider when making our decision to move to the admin-based population estimates (ABPEs) as official population estimates. These are centred around 10 main areas:

• meeting the priority needs of users

• assurance of methods

• data supply and quality

• accuracy and reliability

• coherence and comparability

• systems and resource

• timeliness and punctuality

• clarity and accessibility

• continuing commitment to improvements over time

• clear path towards becoming accredited official statistics

Meeting the priority needs of users

Criterion - We have evidence that the ABPEs will meet priority needs of users.

Meeting user needs is an essential consideration for producers of official statistics. Population estimates are fundamental to a wide suite of functions in government and beyond. We must be confident that we fully understand user needs and that users welcome and are satisfied with our proposals.

Information on user needs has been gathered through a range of engagement over time, giving us an understanding of user needs and the means to assess this criterion. The majority of user needs were obtained through our 2023 Consultation on the future of population and migration statistics. More recently, we learned about user needs through extensive local authority consultations. The engagement activities carried out in the process of creating these criteria also provided further insight into user needs.

User needs have been crucial in the identification and development of the ABPE methods. For example, the ABPEs should produce a consistent demographic account of the population by age and sex by different levels of geography.

It is also important that there is an extensive evidence base to draw on. This will ensure that user needs are met in future developments. We have a defined process for understanding and recording user needs for the ABPEs. The ABPEs should be more flexible to respond to user needs than traditional methods. However, user needs are varied, and we will not be able to meet all needs. We will be transparent about where the ABPEs cannot meet all the needs identified, through our continued engagement with users.

Assurance of methods

Criterion - We have comprehensive and transparent assurance structures in place, in line with our best practice, to ensure that our statistical methods are fully scrutinised by experts.

Criterion - We must seek and respond to advice on methods from the assurance bodies we have in place.

Criterion - We will publish comprehensive quality information to support the ABPEs.

The methods used to produce the ABPEs are complex. It is important that these methods receive proper scrutiny from our own experts and external academic experts. This will provide confidence that they function correctly throughout the statistical process and provide the best possible estimates.

Methodological expert advice for the Future of Population and Migration Statistics (FPMS) programme is provided by a Methodological Assurance Review Panel (MARP). Further specialist scrutiny, under the oversight of MARP, will be carried out by a sub-group that has greater expertise in Bayesian methods. The group will be in place before the estimates are published in summer 2025. This will ensure that all methods are reviewed by experts with the relevant specialised skillset. This assurance structure will continue to operate once the ABPEs have become the official estimates of the population.

Our action plan published on 28 October 2024 included responses to an independent review of the ABPEs that the Office for Statistics Regulation (OSR) commissioned from Professor Arkadiusz Wiśniowski from the University of Manchester. These responses answered important technical points about the ABPE methods.

The methods used for the ABPEs have been shared in the public domain in our Understanding mid-year ABPEs for local authorities in England and Wales methodology, our Mid-year ABPE for England and Wales quality and methodology information (QMI), and a technical report.

Further outputs are planned, including a methods guide expected around the time the ABPEs are published in summer 2025. The underlying code will also be released, along with supporting technical guidance. There will also be an academic article to be submitted before summer 2025. These outputs will continue to support transparency around the methods, opening them up to wider scrutiny and understanding.

In summary, the expert oversight of the ABPE methods currently in place will be strengthened further, while the extensive explanatory and supporting material already published will be supplemented by future publications. These activities will provide a high level of transparency and assurance of the methods used to produce the ABPEs.

Data supply and quality

Criterion - We have confidence that the administrative data sources we need will continue to be available.

Criterion - The quality of the administrative data meets our needs.

Reliable supply of high-quality input data is essential to produce the APBEs. A strength of the ABPE methodology is that it draws on a range of sources, as described in our Understanding mid-year ABPEs for local authorities in England and Wales methodology. This means that it is resilient, in principle, to the absence of any individual source. However, a secure supply overall remains essential to reliably produce the ABPEs, and the supply needs to be maintained at an acceptable quality.

The supply of data to the Office for National Statistics (ONS) is supported by the Statistics and Registration Service Act and other associated legislation. However, it remains a substantial task to ensure the required supply continues, and that we can accommodate changes to supply. This task is enhanced as opportunities arise to incorporate new or improved supplies into our methods.

We have adopted an internal framework for assessing and monitoring the maturity of the ABPE data supplies to meet these challenges. The framework covers the robustness of data sharing agreements, the effectiveness of supplier communications, the maturity of data transfer systems, and quality assurance procedures. The framework also assesses whether the data sources have any associated quality concerns to address, including the timeliness of the supply. Where risks remain, we will demonstrate the mitigations and plans to reduce the risk in the future.

We monitor the quality of all data sources as standard practice. However, administrative sources are (by definition) not collected for statistical purposes. They require specific monitoring of their quality for our purposes. Our approach to assessing the quality of administrative data sources is covered in our The future of population and migration statistics in England and Wales quality strategy. It includes the use of the UK Statistics Authority's Administrative Data Quality Assurance Toolkit, which provides a framework for quality assuring administrative sources. We expect to publish information on the strengths and limitations of the administrative data sources used in production of the ABPEs, alongside the publication of the estimates in 2025.

There are ongoing initiatives that are relevant to the criteria and are informed by our maturity framework. This includes work to automate the acquisition, handling, and provision of administrative data, which is adding resilience and efficiency to the production of the ABPEs. We are also working with data suppliers on known quality concerns, including improvements to the timeliness of data. The success of these developments will give us the evidence that the criteria can be met.

Accuracy and reliability

Criterion - We have confidence that, on average, the ABPEs will show at least the same accuracy or improved accuracy as the traditional-methods mid-year estimates (MYEs).

Criterion - We will publish comprehensive quality information to support the ABPEs. 

Accuracy is fundamental to the value of the ABPEs and their uses across policy and analytical needs. Its two main components are precision and bias. The traditional method of producing annual mid-year population estimates, benchmarked with a census every 10 years, results in reduced precision and increased bias between censuses. This affects the data in some local authorities more than others.

The reduced precision and increased bias between censuses means that population estimates produced many years after a census have lower accuracy than those produced immediately following a census. The ABPEs exhibit less drift over the 2011 to 2021 intercensal period than the MYEs and a smaller decrease in precision, so they provide a more consistent level of accuracy. This is described in our Transforming population statistics, comparing 2021 population estimates in England and Wales article.

Initially, we aim for the accuracy of the ABPEs to be equivalent to the level of accuracy typically achieved by the traditional-methods MYEs at the mid-point between two censuses. This involves comparing the original MYEs in 2016 with the rebased estimates for the same time period, as a measure of accuracy. The ABPEs for local authority estimates by age and sex will also be compared with the rebased estimates. We will carry out an assessment to determine if they are of at least equivalent quality.

As administrative data and methods mature, we expect the accuracy of the ABPEs to become closer to the level of accuracy achieved shortly after a census. We can assess accuracy of the ABPEs produced for years before Census 2021. However, assessing the accuracy of the estimates for years after 2021 (the most recent census year) is more difficult.

A suitable method for benchmarking the population stock is critically important for ensuring the accuracy of the population estimates, both for the traditional-methods MYEs and for the ABPEs. It is essential that this method allows us to adjust the admin-based estimates to ensure accurate coverage of the population. This will allow us to provide an approximately unbiased stock estimate at regular intervals. We are investigating options for a coverage adjustment method independent of a census.

When we reach our decision point, we will assess our confidence in this work to produce a suitable method. This means that we have identified a suitable method, have consulted experts, and have a plan for delivering these new methods.

Coherence and comparability

Criterion - We will publish our plans for how revisions will be managed in the admin-based population and migration system.

Criterion - We will be open about areas of non-coherence and have plans in place to address them.

Revisions

A publication schedule will support the delivery of timely estimates and will allow for updated and improved estimates to be released under a managed and transparent process. This information is important for users in planning their own use of the data. Some users require early provisional data for their purposes, whereas others would prefer to wait until the estimates are finalised.

Administrative sources can be updated and more data become available, which may result in revisions of previously-published estimates. Revisions ensure initial timely data are available. However, their accuracy is improved when more data become available or there is an improvement to a method.

We are aiming to publish the Publication schedule for admin-based population and migration statistics on 26 February. This article aims to provide more guidance on how we intend to implement the population and international migration statistics revision policy going forward.

Coherence

The ABPEs are designed so that they will be fully coherent in the long term with components of change, like births, deaths, and migration. This is a longstanding user need. However, the ABPEs will lead to several areas of non-coherence with other published population and migration statistics in the short term.

Slightly different methods are already used to compile estimates for England and Wales than for Scotland and Northern Ireland. The move to using ABPEs for England and Wales is likely to increase this incoherence initially.

The move to ABPEs as the official population estimate may also lead to greater incoherence initially between different measures of international migration. For instance, differences are likely to arise between migration as a measure of population change at the England and Wales level (previously published as part of the overall population estimates) and UK estimates of long-term international migration (LTIM).

LTIM is used as an input to the model and is balanced against other sources of information to produce a coherent set of population stocks and flows, producing the ABPEs. This balanced estimate of international migration is likely to differ from the official LTIM estimate, particularly when other sources of information, such as population stocks, provide a different account of underlying populations at local levels. More research is planned in this area to better understand the discrepancies between these two measures, before considering how to align all measures of international migration.

Areas of incoherence within the data are expected. They will be fully declared in supporting published information, like our Mid-year ABPEs for England and Wales QMI. These areas of incoherence are expected to reduce over time as methodological work continues.

Systems and resource

Criterion - We have systems and capability in place to produce the ABPEs and supporting information as part of our regular business cycle.

We have comprehensive contingency measures and risk management procedures in place. Well-designed and resilient systems will enable us to produce the ABPEs reliably and with minimal risk.

The processes and modelling involved in creating the ABPEs are complex and require considerable statistical and technical skills. The systems must be fully tested. Any risks identified through system testing will be addressed through a formal process, including documentation, mitigation, and necessary actions.

Both systems and capability must be assured and backed up with suitable contingency plans. Appropriate business continuity plans and risk management must be in place, to provide contingency measures and minimise individual points of failure.

Evidence we can use to determine that this criterion is met includes:

• appropriate business plans with contingency measures and risk management in place, following our standard practices
• any risks identified in systems testing are formally documented, mitigated, and there are contingency plans in place if the risk occurs
• the development of Reproducible Analytical Pipelines (RAPs)

Timeliness and punctuality

Criterion - We have confidence that priority user needs for the timing of population statistics will be met, in terms of the gap between the reference period and publication.

Criterion - We have confidence that we will be able to meet the publication schedule at least as reliably as currently.

The ABPEs must provide for diverse user needs. For some users, rapid publication (a high level of timeliness) is important, even at the cost of certainty. Early provisional figures can meet this need. Other users require a higher degree of certainty, at the cost of timeliness. The ABPEs production and revision schedules must provide estimates that meet these different user requirements.

It is important that publication schedules are reliably met. This means the risks arising from the additional data requirements and complex methodology inherent to producing the ABPEs are identified and addressed through well-managed contingency plans.

Evidence that the ABPEs meet the criteria for timeliness and punctuality include:

• the ABPEs production schedule is published, as planned, and provides estimates to timetables that meet user requirements
• contingency plans are in place to manage the production process effectively, to ensure that publication schedules are reliably met

Clarity and accessibility

Criterion - Outputs are planned in line with accessibility requirements and our policy, to meet the needs of different user types.

Criterion - Methodology information is available and updated, in a variety of formats to reach a wide range of users.

Outputs need to be accessible and meet the needs of different user types. Our users range from expert analysts to inquiring citizens. Expert users want data quickly, with clear signposting to revisions and methodology information. Inquiring citizens prefer written reports that provide an overview of trends in a visually engaging way. All users must be able to access the supporting information they require to ensure their needs are fulfilled. Much of this is currently available. However, plans need to be in place to ensure access to supporting information continues as the ABPEs become the main measure of population.

We plan to provide outputs that meet different user needs. For example, we will make data available in a data explorer at the same time as the publication of Excel workbooks. We will publish credible intervals to provide metrics on the levels of uncertainty associated with the estimates, which will enable users to interpret the figures effectively.

Continuing commitment to improvements over time

Criterion - The methods and processes supporting the ABPEs can incorporate new sources and methodological improvements as these are developed over time.

The ABPE methodology draws on a range of sources and can take advantage of improvements in available data. Similarly, the methods employed by the model have the potential for further development. It is important that these capabilities are built into the system. A consistent set of principles and good practice are being embedded into the system's coding to support this requirement, using Reproducible Analytical Pipelines (RAPs).

We are developing plans for products that further enhance the value of population statistics, such as estimates of special population types like daytime and seasonal populations. The systems used to produce the ABPEs must be capable of incorporating such improvements into the wider suite of population outputs. These will be supported in practical terms by systems that can adapt to new and evolving data sources.

These capabilities are central to the dynamic population model (DPM) that underpins the ABPEs. We are developing a strategy on the incorporation of new sources.

Clear path towards becoming accredited official statistics

Criterion - We are confident that there is a clear path towards becoming accredited official statistics, and a plan for addressing any outstanding requirements.

Accreditation is important to demonstrate the ABPEs meet the very highest standards for statistical products of trustworthiness, quality, and value. There must be evidence that accreditation is achievable and that there are clear plans for how and when requirements will be met.

Ongoing conversations with the OSR will inform our decision on whether we meet the required standards in the Code of Practice. When we are confident we meet the requirements, we will ask OSR to officially assess the ABPEs as accredited official statistics. We are working to meet the standards expected of accredited official statistics by summer 2025.

The criteria outlined in this article captures what is needed for the admin-based population estimates (APBEs) to be a suitable replacement for the traditional-methods mid-year estimates (MYEs) as the official estimate of population. They have been developed by engaging with users to determine the essential requirements of the ABPEs.

We have defined 17 criteria, centred around 10 main areas, and have provided context and background, where appropriate. We have reported on our progress to date and discussed relevant future work plans, where applicable.

We will assess whether these criteria have been met and publish the outcome before the mid-2024 population estimates are published in summer 2025.

Admin-based population estimates

Admin-based population estimates (ABPEs) are produced by bringing together a range of administrative and other data sources and applying statistical modelling techniques. The statistical model is called the dynamic population model (DPM). It uses available information on the usual resident population (stocks), and movement into and out of the population (flows), at specific points in time. These are official statistics in development.

Administrative data

Administrative data are collections of data maintained for administrative reasons. These may be registrations, transactions, or record-keeping, for example. They are used for operational purposes and their statistical use is secondary. These sources are typically managed by other government bodies.

Coverage adjustment

Coverage adjustment is a method that adjusts the population estimates to account for the fact that some administrative sources will include people who are not usual residents and others may be missed. Coverage adjustment is essential to produce accurate population estimates.

Dynamic population model

A dynamic population model (DPM) is a statistical modelling approach that uses a range of data to measure the population and population changes in a fully coherent way.