Evaluating the accuracy of the admin-based population estimates for England and Wales

• This methodology article accompanies our latest assessment of the admin-based population estimates (ABPEs) against our acceptance criteria, which concludes that the ABPEs will not be adopted as official estimates, given the context of a census in 2031 and the assessment that we would not fully meet the criteria.

• This methodology article is being published to make evidence considered in this decision available to users.

• We assessed accuracy and quality in terms of bias and precision, and we compared a version of the ABPEs without Census 2021 data with the rolled-forward and rebased mid-year population estimates (MYEs) in 2016 (the mid-point between our last two censuses) and in 2021 (the latest census year).

• The ABPEs without Census 2021 data were generally more biased than the rolled-forward MYEs at the national level, but less biased when disaggregated, for example, at the local authority (LA) level or LA by age and sex; further work is needed to fully understand why.

• The ABPEs without Census 2021 data were more precise than the MYEs in 2016, and we could not assess precision in 2021 because of data limitations.

• This evaluation also has important limitations, namely: the decade before 2021 is not a perfect proxy for the current decade; the rebased MYEs are an imperfect benchmark for measuring bias; and the ABPE methods have been under constant development, therefore those used to produce the ABPEs without Census 2021 data here are not fully up to date.

Evaluating the accuracy of the ABPEs by comparing them with our official mid-year population estimates

Following a review against the published acceptance criteria, and the announcement of a census in 2031, we have concluded that the established methods should continue to be used for mid-year population estimates. The need to ensure that population estimates can be produced reliably, with efficient and sustainable systems and processes, means we will continue to use established methods. Admin-based population estimates (ABPEs) are not being adopted as official estimates. This methodology article is being published to share the evidence considered in this decision with users.

This methodology article presents a research evaluation of the accuracy of the ABPEs. It is a simulation study to show us how well the ABPEs would have performed in the decade between our last two censuses (in 2011 and 2021) if we had the methods and ability to produce them following the 2011 Census.

We undertook this work in early 2025, to inform our Assessment of criteria for moving to admin-based population estimates as official estimates of population, England and Wales: 2025. In our 2025 assessment, we shared the overall finding that the ABPEs appear to exhibit less bias and greater precision than the rolled-forward mid-year population estimates (MYEs). We also committed to publishing our work evaluating the ABPEs' accuracy in full.

Early plans for this work were first described in our Criteria for moving to admin-based population estimates as official estimates of population. This set out 17 criteria and focused on 10 areas to consider before the ABPEs could become our official population estimates. The main conclusion from our recent Assessment of criteria for moving to admin-based population estimates as official estimates of population, England and Wales: 2026 was that adopting wholesale methodological changes would not be a sustainable approach. This is considering the context and progress in some areas against the criteria, for example, data supply and quality.

One of the 10 areas was "accuracy and reliability", with accuracy being the focus of the evidence presented in this methodology article.

Evaluation datasets

We use a version of the admin-based population estimates (ABPEs) that excludes Census 2021 data and compare it with our current system based on the mid-year population estimates (MYEs). This means running the dynamic population model (DPM), a Bayesian statistical model that creates the ABPEs, using only data based on the 2011 Census. We also use independent population stocks for each year, and yearly counts of births, deaths, and migration into and out of each area. See Section 7: Glossary for more on the DPM.

We refer to mid-year estimates without Census 2021 data as the "rolled-forward MYE". This is because they were rolled forward from the 2011 Census, using a cohort component approach with yearly counts of births, deaths, and migration. We are evaluating whether the ABPEs without Census 2021 data are at least as accurate as this version of the MYEs, which also excludes Census 2021 data.

We use another dataset, the rebased mid-year estimates, as our benchmark for comparison with both rolled-forward MYEs and ABPEs without Census 2021 data. It represents our best estimate of the population in the decade from 2011 to 2021. The rebased MYEs include both the 2011 and 2021 census data, as described in our Rebasing of mid-year population estimates following Census 2021, England and Wales article.

For more information on these datasets, see Section 8: Data sources and quality.

Why we evaluated data from the decade before 2021

We have focused on the period before Census 2021 because we do not have a benchmark population estimate after 2021. For the current decade, from 2021, it is likely we will have to wait until after Census 2031 for a new benchmark. Having a benchmark is essential for estimating bias, and this is an important part of evaluating accuracy.

For more information on how the current series of mid-year population estimates (MYEs) and ABPEs, based on Census 2021 data compare, see our Admin-based population estimates, local authority case studies, England and Wales, mid-2024 article.

In our Criteria for moving to admin-based population estimates as official estimates of population, we described accuracy in terms of two main components. These components are:

• bias, or the systematic under or over-estimation of the population

• precision, with respect to random errors that arise from sampling, measurement, or statistical modelling

This understanding of accuracy is slightly different to a standard statistical definition, which would focus primarily on bias and would consider precision as another important component of an estimate's quality.

As time passes since the last census, the quality of the mid-year population estimates (MYEs) declines. The estimates become increasingly biased and less precise. Our original objective was for the accuracy of the admin-based population estimates (ABPEs) to match that of the rolled-forward mid-year population estimates (MYEs) at the mid-point between two censuses (in 2016). This meant comparing the rolled-forward and rebased MYEs in 2016, to set our quality benchmark, then assessing whether the ABPEs met or exceeded this standard.

Although we examined both bias and precision, we have placed more emphasis on bias in the analysis we present here.

Time reference period

We have examined bias in the admin-based population estimates (ABPEs) without Census 2021 data and rolled-forward MYEs at the national level across the full range of years from 2011 to 2021.

However, for more detailed comparisons including local authority (LA), age, and sex, we focused on two years:

• 2016 – the mid-point between the 2011 Census and Census 2021

• 2021 – the point of maximum possible "drift" before the latest census, in 2021

We have only been able to examine precision for 2016, because of data limitations. For more details, see Section 5: Comparing the precision of the admin-based population estimates without Census 2021 data and rolled-forward mid-year estimates and Section 8: Data sources and quality.

Our current population estimates, the mid-year population estimates (MYEs), take the population estimates from each decennial census (once every 10 years) as their base.

One of the known challenges with the MYEs is that our estimates "drift" between census years, and their quality declines as more time passes.

This happens at all levels of aggregation: national, local authority (LA), and LA by age and sex. In this section, we examine the relative bias in the admin-based population estimates (ABPEs) without Census 2021 data and MYEs at all these levels: national, LA, and below-LA.

National-level results

All population estimates examined show substantial growth in the population of England and Wales between 2011 and 2021. Compared with our benchmark estimate, the rebased mid-year population estimates (MYEs), the rolled-forward MYEs tend to overestimate the population slightly and the admin-based population estimates (ABPEs) without Census 2021 data tend to underestimate slightly.

Figure 1: All estimates used in this evaluation show consistent population growth between the last two censuses in 2011 and 2021

Estimates of the total population, England and Wales, 2011 to 2021

Notes

1. The rebased mid-year estimate (MYE) is our official population estimate for all years from 2011 to 2021. None of the other population estimates presented above are official statistics.
2. The rolled-forward MYE in these years has been superseded by the rebased MYE, and the ABPEs without Census 2021 are research-only outputs presented for the purpose of this evaluation only.

Figure 1 presents two estimates of the ABPEs without Census 2021 data, alongside the rolled-forward and rebased MYEs. This is because we ran the dynamic population model (DPM) twice without Census 2021 data.

We ran the DPM once from 2011 to 2021 to give us a full series of estimates and the most representative simulation of an ABPE for 2021, without Census 2021 data. We ran it again from 2011 to 2016 only, to produce the most representative simulation of what an ABPE for 2016 would have looked like if we had estimated it at the time.

Although the input data and methods are the same for both series of ABPEs from 2011 to 2016, estimates for these years differ. Including information from later years (post-2016) in the DPM affects the estimated population for 2016 and before. This is the source of the differences between our estimates from 2011 to 2016 in Figure 1 and Figure 2, and why we ran the model twice without Census 2021 data. For more information, see the "Admin-based population estimates" subheading in Section 8: Data sources and quality.

Figure 1 shows how both the rolled-forward MYE and the ABPEs drift away from our best estimate of the "true" population, the rebased MYE. This drift affects some local authorities and age-sex groups more than others, so we have focused most comparisons on these lower, or disaggregated, levels.

Figure 2: The ABPEs are generally more biased than the rolled-forward MYEs at the national level for the total population of England and Wales

Relative bias in the ABPE without Census 2021 data or rolled-forward MYE, compared with the rebased MYE, England and Wales total, 2011 to 2021

Notes

1. A positive value means the evaluation estimate is higher, and negative value means the evaluation estimate is lower than the benchmark, the rebased mid-year estimate (MYE).
2. Relative bias is expressed as percentage change from the benchmark (rebased MYE).

Figure 2 shows that, although the rolled-forward MYEs display less bias across most years, this changes in 2020, where the ABPEs without Census 2021 data are slightly less biased. In 2021, both estimates are approximately equally biased, but in different directions, with the rolled-forward MYEs overestimating the population and the ABPEs underestimating it.

Figure 1 showed that growth in the rebased MYE reduced slightly in 2020, whereas the rolled-forward MYE continued to rise steadily. The reduction in growth in the rebased MYE moves that line closer to the ABPE line, accounting for the ABPEs' lesser bias in these last two years.

Different patterns in England and Wales

We also compared the total population in the ABPE without Census 2021 data and the rolled-forward MYE with the rebased MYE in England and Wales separately.

Figure 3: The ABPEs without Census 2021 data show a different pattern of bias in Wales than in England

Relative bias in the ABPE without Census 2021 data or rolled-forward MYE, compared with the rebased MYE, England and Wales separately, 2016 and 2021 only

Notes

1. A positive value means the evaluation estimate is higher, and negative value means the evaluation estimate is lower than the benchmark, the rebased mid-year estimate (MYE).
2. Relative bias is expressed as percentage change from the benchmark (rebased MYE).
3. The 2016 ABPEs presented here are from the run of the dynamic population model (DPM) from 2011 to 2016 only. This gives us the best simulation of what an ABPE for 2016 would have looked like if we had estimated it at the time. For more information, see "Admin-based population estimates" in Section 8: Data sources and quality.

Figure 3 shows that the rolled-forward MYEs in England are less biased than the ABPEs without Census 2021 data, for both 2016 and 2021. Additionally, the ABPEs always underestimate the population of England when compared with the rebased MYE.

For Wales, we observe a different pattern. The ABPE without Census 2021 data is slightly more biased in 2016 and slightly less biased in 2021 than the rolled-forward MYE. Unlike in England, where the ABPE without Census 2021 data consistently underestimates the total population, in Wales, both these ABPEs and the MYE consistently overestimate the population, and this bias is larger.

Local authority (LA)-level results 

LA-level bias over time: 2016 and 2021

We also examined bias below the national level, starting with local authorities.

Figure 4: At the LA level, population estimates from the ABPEs and rolled-forward MYEs both drift between censuses, but the MYEs drift slightly more

Relative bias in the ABPE without Census 2021 data or rolled-forward MYE, compared with the rebased MYE at LA level, England and Wales, 2016 and 2021

Notes

1. A positive value means the evaluation estimate is higher, and negative value means the evaluation estimate is lower than the benchmark, the rebased mid-year estimate (MYE).
2. Relative bias is expressed as percentage change from the benchmark (rebased MYE).
3. The 2016 ABPEs presented here are from the run of the dynamic population model (DPM) from 2011 to 2016 only. This gives us the best simulation of what an ABPE for 2016 would have looked like if we had estimated it at the time. For more information, see "Admin-based population estimates" in Section 8: Data sources and quality.

Figure 4 shows the swarm plot comparing admin-based population estimates (ABPEs) without Census 2021 data and rolled-forward mid-year population estimates (MYEs) with the rebased MYE, at local authority (LA) level. This is for the 331 LAs that existed in 2021. Consistent with what we know about estimating the population over the decade between each census, it shows the widening range of relative bias in the LA-level estimates between 2016 and 2021, as they drift further from the rebased MYE benchmark.

In 2016, the LA-level ABPEs are within about plus or minus 8 percent of the rebased MYE. The rolled-forward MYEs have a wider range of relative bias, from about 8 percent lower than the rebased MYE to 13.5 percent higher. By 2021, both ranges are larger, with LA-level relative bias in the ABPEs ranging from negative 9 to positive 14 percent in 2021 (compared with the rolled-forward MYE's range from negative 14 to positive 33 percent).

We also calculated the median absolute relative bias (ARB), to summarise these differences in one figure. For more on median ARB, see Section 8: Data sources and quality.

Table 1 shows that the ABPEs without Census 2021 data have a smaller median ARB than the rolled-forward MYEs in 2016 and 2021. This means that the ABPEs are, on average, less biased than the rolled-forward MYEs at LA level.

Local authority (LA) level bias by age and sex in 2016 and 2021

Examining the relative bias in the ABPEs without Census 2021 data and rolled-forward MYEs, broken down by age and sex, can offer additional insights into where bias tends to cluster in each set of estimates.

Figure 5: At LA level, the ABPEs without Census 2021 data exhibit less bias than the rolled-forward MYEs across most single years of age

Distribution of relative bias comparing the ABPE without Census 2021 data or rolled-forward MYE with the rebased MYE at LA level by single year of age, separately for females and males, England and Wales, 2016 and 2021

Notes

1. Relative bias is expressed as percentage change from the benchmark (rebased MYE).
2. The 2016 ABPEs presented here are from the run of the dynamic population model (DPM) from 2011 to 2016 only. This gives us the best simulation of what an ABPE for 2016 would have looked like if we had estimated it at the time. For more information, see "Admin-based population estimates" in Section 8: Data sources and quality.

Figure 5 summarises the distribution of relative bias across 331 local authorities, by single year of age and sex in 2016 and 2021, comparing the ABPEs without Census 2021 data or rolled-forward MYEs with the rebased MYEs.

We see the largest relative bias and widest range of LA values by age and sex generally clustered among young adults (aged approximately 18 to 30 years). We also see similar patterns of difference between the MYEs and ABPEs, when compared with the rebased MYE benchmark. This is what we expect, as young adults are more likely to move around, and are known to be difficult to estimate.

Males aged 90 years and over were also typically outliers in the ABPEs without Census 2021 data. Figure 5 shows that the ABPEs noticeably overestimated the number of males aged 90 years and over in most LAs in 2016, whereas the rolled-forward MYEs are closer to the rebased MYEs in this group. By 2021, however, this group appears to be overestimated in both the ABPEs and rolled-forward MYEs.

A few things could contribute to this. The male population aged 90 years and over is a very small group. This means that any data errors in this age group will be more obvious, as they are a larger proportion of the group.

This would affect both the ABPEs without Census 2021 data and the rolled-forward MYEs. In the ABPEs, the administrative population stock data that feeds into our estimates is known to be of sub-optimal quality for these oldest males and likely contributes to the over-estimation of this group. In the MYEs, small errors, particularly from the census base, may be revealed over time. As the population is "rolled-forward" from one year to the next, errors become a larger proportion of the population as the "true" population becomes smaller over time.

Finally, another noticeable pattern in 2021 is that the ABPEs without Census 2021 data exhibit a dip in coverage relative to the rebased MYEs among females in their early 60s and among adults in their mid-70s. The cause is not entirely clear, and this pattern is not visible in the rolled-forward MYEs.

To summarise differences across all LAs by age and sex groupings, we again calculated and compared the median ARB for the ABPEs and MYEs.

Table 2 shows that the ABPEs without Census 2021 data have a smaller median ARB than the rolled-forward MYEs in both 2016 and 2021. This is consistent with our findings at LA level. It means that the ABPEs are also less biased, on average, than the rolled-forward MYEs when broken down by single year of age, sex, and LA.

Summary of comparative bias and variance in the ABPE and MYE

In addition to median ARB, we calculated the root mean square deviation (RMSD) for the ABPEs without Census 2021 data and rolled-forward MYEs, compared with the rebased MYE. RMSD is a useful summary measure of accuracy that captures both bias and variance. See Section 8: Data sources and quality for more information on calculating RMSD.

A lower value RMSD indicates a smaller difference on average between the evaluation estimate (ABPE without Census 2021 data or rolled-forward MYE) and the benchmark (rebased MYE). Table 3 shows that the ABPE without Census 2021 data has a lower RMSD at LA level and below LA level (by LA, age, and sex) than the rolled-forward MYE in 2016 and 2021.

The root mean squared difference between the ABPE without Census 2021 data and the rebased MYE in 2016 is 3,348 people per LA (smaller than the RMSD of 4,100 people per LA for the rolled-forward MYE in 2016). Similarly, the ABPE in 2021 has a lower RMSD than the rolled-forward MYE.

Table 3 also shows that, in 2016, the RMSD between the ABPE without Census 2021 data and rebased MYE is 56 people, in each LA by single year of age and sex group. This is smaller than the rolled-forward MYE's RMSD of 74 people. In 2021, the RMSDs rise for both the ABPEs and MYEs, but are still lower for the ABPEs than the MYEs (83 and 122 people, respectively).

The ABPEs without Census 2021 data have a consistently lower RMSD than the rolled-forward MYEs, when compared with the rebased MYE benchmark.

We have defined precision as the estimated uncertainty around our point estimates of the population. This means we compared the absolute and relative widths of the 95 percent confidence intervals for the rolled-forward mid-year population estimates (MYEs) and the 95 percent credible intervals for the admin-based population estimates (ABPEs) without Census 2021 data. These tell us how precise we think our point estimates of the population are. For more information on the definition of confidence or credible intervals, see Section 7: Glossary.

Comparing precision in 2016

In this section, we focus on 2016 only and compare the ABPE credible intervals with the MYE confidence intervals at a disaggregated level, for groups defined by: single year of age, sex, and local authority (LA). We compare the absolute and relative widths of the uncertainty intervals around the ABPEs and MYEs for each of the 58,140 LAs by single year of age by sex groups. See Section 8: Data sources and quality for information on these calculations.

The ABPE uncertainty interval was narrower, or more precise, in 97.6 percent of cases (56,763 of 58,140 groups defined by age, sex, and LA). The relative width of the uncertainty interval around the ABPEs (calculated as the width of the uncertainty interval divided by the population estimate), was smaller in 97.7 percent of cases (56,816 groups). From this, we conclude that the ABPEs are more precise than the MYEs.

In the current MYE estimation method, we estimate uncertainty for the population in the latest year by adding the uncertainty of all migration components within the year to the uncertainty of the population estimate in the previous year. In this way, uncertainty accumulates over time, and most of the uncertainty in the MYEs is from the census base and internal and international migration. See our Measures of statistical uncertainty in ONS local authority mid-year population estimates: 2011 to 2019 article for more information.

The uncertainty of ABPEs is obtained from the estimated posterior distribution of the full account from 2011 to the current year, given data on demographic rates, flows, and population stocks (including MYEs in 2011), and their respective uncertainty. See our Understanding mid-year admin-based population estimates for local authorities in England and Wales methodology article for more information on ABPE uncertainty. Pooling data over time and using population stocks makes the increase of uncertainty over time in ABPEs more modest than in the current MYEs.

Finally, estimating uncertainty is an in-built part of the ABPE method. This differs from the MYE method, which only produces population estimates, so uncertainty must be estimated separately.

Important data constraints on precision comparisons

We only have uncertainty (precision) estimates for our ABPEs without Census 2021 data at the disaggregated level. Because uncertainty cannot be summed, we must compare this data with the disaggregated uncertainty estimates for the rolled-forward MYEs.

Disaggregated MYE uncertainty (by single year of age, sex, and LA) was an internal research-only product that was produced for the period from 2011 to 2018 only. Unfortunately, unlike for bias, we do not have the data to repeat our analysis in 2021.

Several additional constraints relate to the groups we were able to compare. The MYE uncertainty estimates were calculated using 2011 LA geographies (348 LAs). Those for the ABPEs were calculated on 2021 geographies (331 LAs), meaning we can only compare the data for the 323 LAs that existed in both 2011 and 2021.

Additionally, ABPE uncertainty was calculated individually for each age from 90 to 105 years, whereas the MYE uncertainty estimates grouped all males or females aged 90 years and over together. After excluding ages 90 years and over from these comparisons, we had 58,140 unique groups to compare, based on 323 LAs multiplied by 90 single-year age groups (0 to 89 years), multiplied by two sexes.

Finally, credible intervals (for ABPEs) and bootstrapped confidence intervals (for MYEs) are both uncertainty estimates. However, they are not perfectly equivalent statistical comparisons, so caution is needed when interpreting these findings.

The analysis we have carried out here was the best we could produce with the data, time, and resources available to us. We have shown that the admin-based population estimates (ABPEs) without Census 2021 data are generally less biased below the national level and more precise than the existing mid-year population estimates (MYEs), using data from 2011 to 2021. However, there are some important limitations to this analysis.

Uncertain benchmark for assessing bias between census years

The census is the gold standard for estimating the population. Although our Criteria for moving to admin-based population estimates as official estimates of population set out the aim of comparing the ABPEs and MYEs in 2016, at the mid-point between the last two censuses, comparisons of bias in 2016 rely on the accuracy of our benchmark population estimate, the rebased MYE.

We are more certain that our mid-year estimates are as accurate as possible in a census year, like 2021, than in the middle of the data series, which is the furthest possible point from a census. For this reason, we have presented our comparisons in both 2016 and 2021. Comparing our estimates in 2021 also allows us to look at the point of maximum possible bias, or "drift" away from the 2011 Census. Our findings for these two years were consistent, which gives us greater confidence.

Another important point is that the rebased MYE is constructed using the MYE that is rolled forward from the 2011 Census, plus information from the most recent Census in 2021. As a benchmark, this means it could be slightly biased against the ABPEs and towards our comparator dataset, the rolled-forward MYE. However, ABPEs generally outperformed the rolled-forward MYEs below the national level in the 2011 to 2021 period. This gives us greater confidence in their improved accuracy.

Applicability of our results to the current decade (2021 to 2031)

We have shown that the ABPEs generally outperformed our rolled-forward MYEs during the last decade ending in 2021. For the current decade, we can only use these findings as a proxy for how the ABPEs and MYEs might perform. This is because there is no benchmark population estimate in the decade following Census 2021 against which to evaluate the current series of ABPEs and MYEs.

Recent demographic and policy changes, and changes to our methods for producing statistics on international migration, may affect the relevance of applying our findings to the current decade.

In addition, Census 2021 took place at an unusual time, during the global coronavirus (COVID-19) pandemic. The pandemic affected patterns of internal and international migration. This means that some people (particularly students and younger workers) might not have been where we would expect them to be in the absence of a pandemic.

Finally, the ABPEs published after 2021 use data more tailored for their production, like the Statistical Population Dataset (SPD). Before 2016, the Patient Register was the ABPE's main population stock, and these data are less optimised for population estimates production. We might therefore expect the ABPEs to perform even better in the current decade than in the period we analysed.

Older ABPE methodology

The methods used to produce the ABPEs with the dynamic population model (DPM) were constantly developed and improved by our methodologists with regular advice from internationally recognised external experts.

The ABPEs without Census 2021 data used in this evaluation are a research-only product, whose methods most closely resemble those used to produce the mid-2023 ABPEs (in summer 2024). They were produced before we began using hierarchical modelling with multiple draws to estimate aggregate uncertainty. This is the main difference between the models used in this work and the methods used to produce the mid-2024 ABPEs from our most recent publication (in summer 2025).

Administrative data

Collections of data maintained for administrative reasons. These may be registrations, transactions, or record-keeping, for example. They are used for operational purposes, and their statistical use is secondary. These sources are typically managed by other government bodies.

Bayesian methods

A statistical approach that updates prior beliefs about an event using new evidence, resulting in a revised belief, or posterior probability. Bayes' theorem is used to calculate the probability distribution of a set of parameters given observed data, integrating both prior knowledge and new observed data. 

Bayesian demographic accounts are a statistical framework for estimating the population and components of population change, incorporating prior knowledge and estimates of stocks, flows, and rates and their uncertainties using a Bayesian approach. All unknown quantities are treated as random variables to reflect uncertainties.

Benchmark estimate

A standard against which to compare other data. In this methodology article, we use the rebased mid-year estimates (MYEs) as our benchmark. This is because it is the best estimate of the population that we have for the period from 2011 to 2021.

Using the rebased MYEs as a benchmark allows us to assess the level of bias in our evaluation datasets:

• the admin-based population estimates (ABPEs) without Census 2021 data (which represent our new methods for estimating the population)

• the rolled-forward MYEs (which represent our existing methods for estimating the population)

See Section 8: Data sources and quality for more information on these data sources.

Cohort component approach

A standard demographic approach that estimates the size of the population using the components of population change to update a population base.

An estimate of the population in the following year, given by the cohort component approach, for example at local authority (LA) level, is obtained by:

• ageing on the previous year's population at the mid-year reference point (30 June) by one year
• adding those who were born during the year
• removing those who died during the year
• adding those who moved into the LA
• removing those who moved out of the LA

Confidence intervals

As used in frequentist statistics, a confidence interval gives an indication of the degree of uncertainty of an estimate and helps to decide how precise an estimate is. It specifies a range of values likely to contain the true population value. These values are defined by lower and upper limits, or bounds. For more on the mid-year population estimates' confidence intervals, see our Methodology for measuring uncertainty in ONS local authority mid-year population estimates: 2012 to 2016.

The 95% confidence intervals for the mid-year estimates (MYEs) used in this evaluation are empirical uncertainty intervals. They are produced using the cohort approach for demographic accounting, and include uncertainty from census, internal and international migration. We assume no uncertainty for birth and death estimates. They are based on the 2.5 and 97.5 percentiles of 1,000 simulations for each set of MYEs by single year of age (SYOA), sex, and local authority (LA). Of these 1,000 simulations, 95% (or 950) are contained within the 95% confidence intervals.

Coverage adjustment

Coverage adjustment is a method that corrects for bias in population stocks. For example, some administrative population stock sources will include people who are not usual residents and others may be missed. Because of this, coverage adjustment is essential to produce accurate population estimates, particularly in the Dynamic Population Model (DPM), which produces the ABPEs. Both our usual ABPEs that include Census 2021 data and the research-only ABPEs without Census 2021 data used for this evaluation, use annual population stocks that are coverage adjusted. The methods to produce MYEs do not use population stocks and therefore do not include coverage adjustment.

All ABPEs currently use a census-based coverage adjustment method. See our Mid-year admin-based population estimates for England and Wales quality and methods guide for more information on coverage adjustment. The coverage adjustment for the research-only ABPEs without Census 2021 data was based only on the 2011 Census.

Credible intervals

The range of values likely to contain the true population value. They are the Bayesian equivalent of frequentist confidence intervals.

The 95% credible intervals for the admin-based population estimate (ABPE) are created by taking the "lower bound", which is calculated as the 2.5 percentile of the distribution produced in the dynamic population model (DPM) estimation process, and the "upper bound", which is calculated as the 97.5 percentile of the distribution. In this case, we can say that the probability that the true value lies in the credible interval is 95%.

Dynamic population model (DPM)

The dynamic population model is a demographic accounts framework that uses Bayesian statistical modelling to create coherent demographic estimates from multiple imperfect datasets. Statistical modelling allows for data inputs with differing levels of bias and precision and takes account of underlying demographic trends, when estimating outputs. The dynamic population model (DPM) is the model used to produce the admin-based population estimates (ABPEs).

Patient Register (PR)

A health dataset, used in multiple population and migration estimates, which is no longer produced (see our Population estimates for the UK, mid-2021: methods guide). It was replaced by the Personal Demographic Service (PDS) dataset.

Statistical Population Dataset (SPD)

A linked administrative dataset, which applies a set of inclusion rules to approximate the usually resident population. See our Developing Statistical Population Datasets, England and Wales: 2021 article for more information.

Measuring the data

Measuring bias

Our main measure of bias to compare the admin-based population estimates (ABPEs) or the rolled-forward mid-year population estimates (MYEs) and the rebased MYEs was relative bias, expressed as a percentage change from the benchmark estimate:

The "evaluation estimate" here refers to either rolled-forward MYEs or the ABPEs without Census 2021 data. Our "benchmark estimate" always refers to the rebased MYEs, which are our official estimates of the mid-year population from 2012 to 2021.

Summary measures for bias

Where we examine differences at lower levels of aggregation (below the national level), we summarise the differences between our benchmark data and the ABPE, without Census 2021 data or rolled-forward MYE, using median absolute relative bias (ARB). This is the median value of:

Summary measures for bias and variance

Root mean square deviation (RMSD) is the square root of the mean squared deviation, where "deviation" refers to the numerical difference between our evaluated estimates (ABPE or rolled-forward MYE) and our benchmark estimates (rebased MYE):

Where:
n = the number of data points or sample size
Y_i = the observed values (data being evaluated)
Ŷ_i = the predicted values (benchmark data)

RMSD translates the "deviation", or difference from the benchmark data, into the original unit of measure. In our case, the unit of measure is people.

Measuring precision

As we have defined precision in terms of uncertainty, we have relied on measures that calculate the absolute and relative width of empirical 95% confidence intervals (for the MYEs) or credible intervals (for the ABPEs), which provide an "upper" and "lower" bound for the point estimates of the population, as follows:

This calculation of the relative width of the uncertainty interval follows the definition laid out in our Indicative uncertainty intervals for the admin-based population estimates: July 2020 methodology article.

We then compared how wide these intervals were for the rolled-forward MYEs and the ABPEs without Census 2021 data.

Admin-based population estimates

We used a research-only version of the admin-based population estimates (ABPEs) that excludes data from Census 2021. This is so that we could simulate the performance of ABPEs relative to the MYEs, had the Dynamic Population Model (DPM) methods been available prior to Census 2021.

We used two research-only versions of the ABPEs without Census 2021 data.

ABPEs without Census 2021 data (2011 to 2021)

Produced from a run of the dynamic population model, or DPM (see Section 7: Glossary for information on the DPM), using data for all years from 2011 to 2021.

This is used for all our bias comparisons in 2021 because it provides the most accurate simulation of what an ABPE in 2021 would have looked like prior to Census 2021 data being available. It was not used for precision comparisons.

ABPEs without Census 2021 data (2011 to 2016 only)

Produced from a run of the DPM using data for all years from 2011 to 2016. This run replicates the ABPEs that would have been produced had the DPM methods been applied as soon as data became available to produce mid-2016 ABPEs. It was important to run this additional simulation, as including additional data beyond 2016 (as in the 2011 to 2021 ABPEs) affects the estimates prior to 2016. These ABPEs provide our primary estimates for evaluating ABPEs for 2016.

Rebased mid-year population estimates (MYEs)

This is our benchmark estimate for assessing bias (see Section 7: Glossary for more information). The rebased MYEs used here include data from both the 2011 and 2021 Census.

Following each census, the MYEs are updated to provide the best estimates of the population in the 10 years between the previous two censuses (for example, between the 2011 and 2021 2021 Census). This is a two-step process called reconciliation and rebasing.

First, reconciliation explores the differences between a new version of the mid-year estimates, based on the latest census, and the existing "rolled-forward" MYE for the latest year (as in 2021). See our Rebasing and reconciliation of mid-year population estimates following Census 2021, England and Wales: 2022 article for more information on this process and why we undertake it. Our Reconciliation of mid-year population estimates with Census 2021 at local-authority level article provides more information on the results of the latest reconciliation.

Second, we use the results of reconciliation to "rebase", or revise, our estimates from 2012 to 2021. These then replace the rolled-forward MYEs as our official population estimates of the population. For more information on this, see our Rebasing of mid-year population estimates following Census 2021, England and Wales article. The rolled-forward and rebased MYE are the same in 2011, as they are heavily based on the 2011 Census. The 2011 MYE is not rebased following Census 2021. It forms the start of both series.

Rolled-forward mid-year population estimates (MYEs)

The rolled-forward mid-year population estimates (MYEs) are our main comparator for the ABPEs without Census 2021 data. For more information on these methods, please see our Mid-year population estimates quality and methodology information (QMI).

The rolled-forward MYEs that we use in this methodology article are based on the 2011 Census only. Unlike the rebased MYEs, they do not include any data from Census 2021, though they integrate information on births, deaths, and migration each year.

They were produced and published for the years from 2011 to 2020, as our official population estimates. Following the rebasing exercise, they were superseded by the rebased MYEs.