Improving international migration statistics using administrative data

The Office for National Statistics (ONS) produces regular estimates of international migration. Our long-term international migration (LTIM) statistics estimate the flow (or movement) of migrants to and from the UK. This includes our headline estimate for net migration, which is the difference between the number of people who enter the UK (immigration) and the number of people who leave (emigration).

To produce these estimates, we use the UN-recommended definition of a long-term international migrant (PDF, 5.0MB), that is: "A person who moves to a country other than that of his or her usual residence for a period of at least a year (12 months), so that the country of destination effectively becomes his or her new country of usual residence."

These estimates are important because migration statistics help describe how the number of people living in different parts of the country changes over time. This helps decision makers to plan and allocate resources effectively, including the funding of local services.

To ensure that we continue to provide accurate and timely statistics that meet users' needs, we have been improving our methods, as explained in our long-term international migration estimates methodology. This includes moving away from survey-based data where we would ask people if they were moving to or from the UK, to using administrative (admin) data. Admin data are data collected for the purposes of services such as tax, benefits, and health systems. This includes data collected at the border as people enter or leave the UK. However, estimating international migration is not as straightforward as counting people in and out of the country, and the reasons why can be found in our Understanding international migration statistics methods guide.

Here we outline how we measure international migration and the data we use. This includes an explanation of our research, the main improvements we have made and future developments. It is important to note that the recent transformation of our methods means that our statistics are currently classified as official statistics in development, as defined by the Office for Statistics Regulation. We have consulted with users about this transformation and, once our methods stabilise, we are considering a change in the classification of our statistics to "official statistics".

Before the coronavirus (COVID-19) pandemic lockdown measures were introduced in the UK in March 2020, we at the ONS relied on the International Passenger Survey (IPS) to collect migration data. The IPS conducts face-to-face interviews with a sample of passengers as they pass through major UK airports, sea routes and the Channel Tunnel. More information on the survey methodology can be found in our IPS Quality and Methodology Information Report.

Prior to the pandemic, the IPS was the best source of data that we held for information on both long-term immigration, emigration and therefore, net migration estimates for the UK. These data are based on the reported migratory intentions of UK residents departing the country and foreign residents arriving. Adjustments were then made to account for asylum seekers, resettlement schemes, changes in intentions, and flows to and from Northern Ireland.

This method of reporting has its challenges; mainly that people can change their plans and so the intentions stated may not reflect actual behaviours. This limitation is more prominent during periods of high uncertainty, such as during the pandemic. Furthermore, our research from 2019, as published in our Understanding different migration data sources article, found that the IPS had been underestimating EU immigration and underestimating non-EU student emigration.

While adjustments were introduced for these groups, the IPS was quickly recognised as being stretched beyond its original purpose. This has been shown in our updated migration estimates, where we compared our historical net migration figures with data from administrative sources, including Census 2021. In this case, adjustments still did not adequately measure migration for these groups and confirms that admin-based migration estimates (ABMEs) are a better estimate than IPS-based estimates. More information can be found in Estimating UK international migration: 2012 to 2021 article.

In August 2019, we asked the Office for Statistics Regulation (OSR) to support our reclassification of international migration statistics to Experimental Statistics (now referred to as official statistics in development). In August 2020, we announced in our Population and migration statistics system transformation article that we would be moving away from using the IPS to produce official international migration statistics. Instead, we would be moving to produce ABMEs. The suspension of the IPS during the coronavirus pandemic accelerated this ambitious programme of work. Since May 2022, when we published our Long-term international migration, provisional: year ending June 2021 bulletin, we have been using our new methods, as outlined in the following section.

Now that we know the challenges of the International Passenger Survey (IPS) data, we have moved to use admin data to identify long-term international migration based on actual behaviours, rather than intentions. We use the wealth of information available within several admin datasets in secure and ethical ways to estimate international migration.

These admin-based migration estimates (ABMEs) bring data together and estimate the likely outcome of long-term international migrants who have recently immigrated or emigrated. Our ABMEs measure EU and non-EU nationals migrating to the UK. For British nationals, we continue to use the IPS while we explore admin sources that we could use to measure this group.  

We currently produce our migration estimates broken down by EU, non-EU and British nationals. The breakdown of EU and non-EU nationals for international migration pre-dates the UK exiting the European Union. Because of historically different rights to reside in the UK (prior to the UK leaving the EU, EU nationals did not need a visa to reside in the UK as long-term residents), these groups have been measured separately to provide a comparison.

In the future we will be looking to further break down these groups and focus on visa holders and people with indefinite leave to remain. For more information on how we produce international migration estimates, you can read our Methods to produce provisional long-term international migration estimates methodology, published in November 2023. This publication contains information on the recent methods used to include asylum seekers and resettled refugees.

In this article, we focus on our estimates of long-term international migration flows (immigration, emigration and net migration). However, our users also want to know about international migrant stocks (the total number of international migrants present in the country at a particular point in time). For more information on the international migrant population present, we have published information from Census 2021. This is outlined in our International migration, England and Wales: Census 2021 bulletin which is our best available data source.

Non-EU nationals

To produce migration estimates of non-EU nationals we use Home Office Borders and Immigration data. These data take an individual's visa information and link it to any corresponding travel into or out of the UK during the individual's visa period.

To identify long-term migration using this information, we first identify individuals who are on a visa that lasts at least a year. Then, using their travel data, we identify individuals who have arrived in the UK and subsequently stayed in the UK during the remaining time on their visa. To estimate the emigration of this group, we identify a departure date at the end of an individual's visa and then check whether they remained out of the UK for a period of at least a year.

The most recent year will not have a full 12 months of data, so we need a different approach. This involves applying an adjustment to the data that accounts for the fact that some individuals will not stay for 12 months after their arrival, and others who have left the UK may return within 12 months.

EU nationals

To measure migration of EU nationals, we currently use the Registration and Population Interaction Database (RAPID), which is explained in our Methods for measuring international migration using RAPID administrative data methodology. This gives us the best estimates of migration with the data available to us, as not everyone from the EU needs a visa to migrate to the UK. Created by the Department for Work and Pensions (DWP), RAPID provides a single coherent view of an individual's interactions with the tax and benefits system; these interactions can then be used to indicate arrival or departure from the UK.

The RAPID database covers everyone with a National Insurance Number (NINo). To work, claim benefits or apply for a student loan, anyone arriving in the UK for the first time needs to apply for a NINo and because of this, the RAPID coverage is extensive.

However, there are some people who might not appear in RAPID because a NINo was not issued. For example, students may be excluded if they are not working or claiming benefits, or if they are children under the age of 16 years who are not eligible for a NINo. We adjust our EU estimates to account for this under-coverage by using Higher Education Statistics Agency (HESA) data linked to HMRC pay-as-you-earn real-time information to adjust for students who do not work alongside their studies. We also use insights from the International Passenger Survey (IPS) on the ratio of EU-nationals aged under 16 years to adults (over the age of 16 years) immigrating to and emigrating from the UK to adjust the RAPID-based estimates.

Since January 2021, EU nationals have been required to obtain a visa to move to the UK, or have EU Settled Status (EUSS). This means that we now have the same data for EU nationals as we do for non-EU nationals. We are continuing research to assess whether the methods for estimating non-EU nationals will also apply to EU nationals. Research is still in the early stages and we have made progress in identifying EU nationals who hold visas.

We have started to apply a similar rationale as for non-EU nationals to decide whether they should be considered long-term immigrants in the UK. The similarity in the methodology used to produce estimates for non-EU and EU nationals increases the comparability between the two. We will compare these estimates with other data sources such as RAPID and address any limitations in this new data source when compared with other sources. This includes addressing the Common Travel Area and the extent to which data are missing from this method of travel, and how this affects migration estimates.

British nationals

Measuring migration of British nationals remains a challenge. Compared with other nationality groups, British Nationals have much more freedom to move in and out of the UK without informing an administrative body, and therefore data are not collected in the same way. We are exploring whether the Registration and Population Interaction Database (RAPID) could provide some potential using similar methods to those for EU nations. However, there are many reasons why a British national could stop interacting with the tax and benefits systems but remain in the UK, for example to study or for a career break. In other words, if British nationals stop interacting with these systems, this does not necessarily mean that they have left the UK.

While we continue to investigate the potential of different sources, the IPS remains our source of information for measuring these migrants. Recent findings suggest that the IPS is underestimating the number of British nationals emigrating, so we are accelerating research to ensure these groups are adequately covered in our net migration estimates.

We are still assessing which data source or combination of data sources will replace the IPS.

The Digital Economy Act 2017 has given us at the ONS increased access to admin data. This means that we can use a wider range of more accurate and timely data to produce our estimates, since admin data from public authorities in the UK can be linked and shared with the ONS and are based on actual interactions rather than stated intentions.

As part of the radical programme to transform population and migration statistics at ONS, we are moving to use admin-based data to collect demographic information, such as information on a person's age and their sex. We now take account of all relevant data sources. This helps us to improve our methods as well as to meet the changing needs of our users so that they can make appropriate use of the data.

We have quality assured our admin-based long-term international migration estimates against many data sources, including Census 2021. This has improved our confidence in the admin-based migration estimates (ABMEs). We have published a paper titled Estimating UK international migration: 2012 to 2021 which is a result of the programme of work we have been doing to assess the quality of our estimates. This paper explains how our ABMEs align closely with the data from Census 2021, and therefore further demonstrates the confidence we can have in admin data compared with the previously used International Passenger Survey (IPS).

The updated estimates also show that survey data underestimated British emigration. All aspects of both immigration and emigration will have an impact on net migration estimates. This provides further evidence that previous adjustments to the IPS have not gone far enough to account for all aspects of net migration. Quality-checking our previous decade of estimates using the IPS against Census 2021 data demonstrates the improvement in measuring international migration using admin data compared with the IPS estimates.

We now have a new, robust methodology for estimating long-term international migration which has been developed in consultation with migration experts and other government departments, as well as being supported by the results of Census 2021.

In the future, we will stop relying on the International Passenger Survey for any of our estimates. Instead, we plan to make increased use of Home Office Borders and Immigration data and wider population statistics produced by the Office for National Statistics (ONS). To ensure our various estimates match up with one another, we will compare multiple admin data sources and draw on the strengths of each.

As we further understand the data and as our methods develop, we will explore how we can continue to progress our work on international migration. This will include providing our users with more detailed information of those who migrate, such as breakdowns by age, sex, and nationality, identifying which local authority an individual migrates to and from, and looking to change the grouping of our estimates from EU/non-EU/British nationals to visa/non-visa groups. This information is important to help decision makers understand the different impacts of policy.

We will be able to use the information we get from admin data to expand our outputs, beyond the estimates of long-term international migration that we currently produce. We are now developing methods, which will provide insights into different types of international mobility, including temporary and short-term migration, as well as diverse patterns of international mobility, for example, seasonal work or circular migration.

We are working to ensure that the code we write to carry out our methods is easy to read and understand and that there is a clear log of the changes made as the methods develop. These developments help improve the quality of our data and analysis. They will also make it easier for others to reuse and reproduce our analysis.

To provide up-to-date data (within five months of the reference period), we initially provide provisional estimates. These provisional estimates provide an early indication of migration until we have seen whether people really did stay in the UK or left the UK for a full 12 months. Initial adjustments are made to the data based on averages from previous years and actual, observed data.

We are working to improve our adjustment method and develop a more refined approach. For more information, see our International migration research, progress update: November 2023 article.

Migration and population estimates

The ONS is researching the use of the Dynamic Population Model (DPM) to produce admin-based population estimates (ABPE). The DPM currently produces ABPEs for the England and Wales population, balancing the size of the population alongside local and national migration to ensure these estimates are coherent. The development of this modelling approach is ongoing.

The classification of our statistics

We have increasing confidence in admin-based international migration estimates and our methodology is beginning to stabilise. We have ambitions for UK international migration statistics to be Accredited Official Statistics, which will eventually include removing the "official statistics in development" label. We will continue to work with Office for Statistical Regulation (OSR) on an appropriate timeline for this. The "official statistics in development" label continues to help our users to understand the stage of research.

For further information or to offer feedback, please email pop.info@ons.gov.uk.

Administrative data

Collections of data maintained for administrative reasons, for example, registrations, transactions, or record keeping. They are used for operational purposes and their statistical use is secondary. These sources are typically managed by other government bodies.

Circular migrant

A person who has crossed the national borders of the reporting country at least three times over a 10-year period, each time with a duration of stay (abroad or in the country) of at least 12 months.

Home Office Borders and Immigration data

Combines data from different administrative sources to link an individual's travel in or out of the UK with their immigration history. This system has data for all non-European Economic Area (non-EEA) visa holders.

International Passenger Survey (IPS)

Our International Passenger Survey (IPS) collects information about passengers entering and leaving the UK and has been running continuously since 1961. The IPS was resumed in January 2021 after being suspended since March 2020 because of the coronavirus (COVID-19) pandemic. Currently, we use it for our British national estimates and for providing information on reason for migration.

Long-term international migration

Long-term international migration (LTIM) statistics estimate the flow (or movement) of migrants to and from the UK. This bulletin uses the UN-recommended definition of a long-term international migrant, as explained in the Recommendations on Statistics of International Migration paper (PDF, 5MB). It is defined as "A person who moves to a country other than that of his or her usual residence for a period of at least a year (12 months), so that the country of destination effectively becomes his or her new country of usual residence."

Net migration

Net migration is the difference between the number of people coming to live in the UK (immigration) and the number of people leaving to live elsewhere (emigration). When more people are arriving in the UK than leaving, net migration is above zero and so adds to the non-UK population.

Registration and Population Interaction Database (RAPID)

Registration and Population Interaction Database (RAPID) is a database created by the Department for Work and Pensions. It provides a single coherent view of interactions across the breadth of benefits and earnings datasets for anyone with a National Insurance number (NINo).