The ONS response to the OSR review of migration statistics: February 2023

The Office for National Statistics (ONS) is transforming how it measures international migration. We have moved away from using the International Passenger Survey (IPS) to estimate migration and are undertaking an innovative programme predominantly based on the use of administrative data.

Given the importance of these statistics and the transformation programme underway, the Office for Statistical Regulation (OSR) conducted and published a review in March 2022 to ensure that the transformed migration statistics are trusted, high quality and of value in line with the Code of Practice for Statistics. This article is an update on our progress against the recommendations raised by the OSR in their Review of Migration Statistics produced by the Office for National Statistics, published on 1 March 2022, one year on.

In summary, the OSR found that we have been credible and robust in our development of the methods and data sources underpinning our transformation of migration estimates and have engaged well with experts on these developments. They recommended that we should build on this by:

• publishing details of the methods and data sources we intend to use, and the underlying uncertainty associated with these

• improving and broadening our user engagement

• ensuring clear and coherent communication of our migration plans in the context of the population and labour market transformations

The review of migration statistics produced by the Office for Statistical Regulation (OSR) and published on 1 March 2022 considered our Long-term international migration, provisional: year ending December 2020 bulletin, published on 25 November 2021. The figures published in this bulletin were based on experimental statistical modelling. Since then, there have been a number of significant improvements. These include:

• methods - improved communication of methods, ensuring they are accessible to a wider range of users, see our Methods to produce provisional long-term international migration estimates methodology

• data sources - increasing predominance of administrative data for the international migration estimation and limited reliance on State Space Modelling (SSM)

• quality assurance - internal and external expert review of our methods as we have progressed the transformation

• user engagement and communication - we have worked closely with our users to develop statistics to meet their needs using a variety of forums to maximise our reach

Methods

Summary of the Office for Statistical Regulation (OSR) recommendations

The Office for National Statistics (ONS) should:

• provide more transparency in methods, so they are easier to understand for less expert users

• provide a clear understanding of uncertainty associated with estimates

• inform users on the long-term plan for measuring migration, including maintaining a consistent time series

Following the suspension of the International Passenger Survey (IPS) during the coronavirus (COVID-19) pandemic, estimates of international migration were estimated using State Space Modelling (SSM) techniques. This approach, upon which the OSR conducted their review, modelled historic IPS data using more timely administrative data (Registration and Population Interaction Database (RAPID) and Home Office Border Force data). This was a tactical solution to ensure users had consistent measures of international migration over the course of the coronavirus pandemic. It was always intended to be a short-term solution while we continued to develop methods that relied more on administrative data, recognising the long-standing shortcomings of the IPS. Estimates using the SSM approach were published in Using statistical modelling to estimate UK international migration and in our Long-term international migration, provisional: year ending December 2020 bulletin.

In May 2022 we published estimates of migration for the period up to year ending June 2021. These were the first to be published predominantly using administrative data (as described in the Data sources sub-section). Observed migration activity, from early insights of provisional census results, provided us with confidence that estimates derived from administrative data sources are more accurate than those derived from SSM-modelled IPS estimates. Additionally, these data and methods produced estimates that were more comparable with latest Home Office statistics on the operation of the immigration system.

We are still researching how to move to admin-based estimates for the much smaller group of British nationals migrating long-term. Until that research is concluded, we have revisited the SSM assumptions to improve the quality of the estimates.

To support users' understanding of these methods, and their progression over time, we have maintained a "plain English" summary in our Methods to produce provisional long-term international migration estimates methodology. This summarises the methods used to produce experimental and provisional estimates of long-term international migration flows, published in our Long-term international migration, provisional: year ending June 2022 bulletin, as well as those preceding it.

We recognise that users need to understand the uncertainty surrounding estimates of migration, especially during this period where the methods are experimental. We have communicated uncertainty in a qualitative manner, for example in Section 3: How these international migration estimates can and cannot be used of a recent bulletin. However, measuring uncertainty quantitatively remains a challenge, having moved away from a survey-based approach.

We have also shared comparisons against alternative indicators, providing users with a sense of quality of the variation around the point estimates in our Long-term international migration, provisional: year ending June 2022 bulletin. We are currently researching methods to produce uncertainty measures for the international migration estimation. We plan to publish a working paper in May 2023 that explains our specific iterative plans to achieve this, and that will present some experimental estimates of uncertainty (variance). We will develop our work in this area as we explore these new methods. We are researching both direct and indirect measures of uncertainty as per recommendations in the OSR's Approaches to communicating uncertainty in the statistical system (PDF, 604KB).

The dynamic population model (DPM) is an important new strategic development for the ONS. The DPM uses statistical modelling to draw strength from a range of data sources, including administrative and survey data, to produce timely estimates of population and population change. Data and system models within the DPM formally apply demographic expertise to the DPM outputs. In the first instance, DPM outputs will be a fully coherent demographic account (population stocks, births, deaths, internal and international migration and cross-border flows) for 2011 to 2022, by age, sex and local authority.

It is envisioned that initial admin-based international migration estimates will be inputs to the DPM. Adjusted admin-based migration estimates will be among the outputs from the DPM, which will be fully coherent with other components of the demographic account, both in a given year and over time.

Data sources

Summary of OSR recommendations

The ONS should publish sources and methods and how they impact estimates, including provisional versus final estimates.

Our current methods are based predominantly on administrative data. Specifically, non-EU migration figures now use Home Office Border Systems data, and EU estimates are based on Registration and Population Interaction Database (RAPID) data received from the Department for Work and Pensions and HM Revenue and Customs. British nationals estimates are derived from the IPS. These sources are augmented with other data to address coverage gaps. For example, to identify EU students immigrating into the UK long-term, we use Higher Education Statistics Authority (HESA) data. We have set out how each additional data source affects the methodology of the production of the international migration estimation in our Methods to produce provisional long-term international migration estimates methodology in November 2022.

An important feature of the transformed migration system is the process of revisions, as estimates are initially provisional before becoming finalised. As described in our recently updated Revisions policy for population and international migration statistics (May 2022) this is a consequence of both improving methods over time, as well data maturing. We are transparent on the extent of revisions, for example in Section 8: Revisions to migration statistics in our Long-term international migration, provisional: year ending June bulletin. Here we highlight the scale of revisions to year ending June 2020 and year ending June 2021, and our expectation that estimates for year ending June 2022 will be revised in May 2023.

As the data we rely on are largely collected outside of the ONS, the quality of migration estimates are at risk if timeliness or frequency of supply is compromised. We actively manage this with our Data Growth and Operations Directorate and through Memorandums of Understanding with the Home Office (HO) and the Department for Work and Pensions (DWP) on secure data shares. The Migration Statistics Steering Group has been helpful in providing guidance, expert advice and quality assurance to progress the development of the international migration estimation. 

At a working level, we have embedded ONS analysts on secondments within the HO and DWP, who have controlled access to their data systems. We have strong relationships with both departments, working together to ensure expert understanding of the data sources that feed into the estimates.

Data pipeline processes are being met by the introduction of Reproducible Analytical Pipelines (RAP) into the production of migration statistics. We have developed a RAP strategy that is underpinned by ONS best practice guidelines.

Quality assurance

Summary of OSR recommendations

The ONS should give users greater confidence in how it approaches quality assurance, and validates migration estimates.

We work closely with experts, including colleagues from the Government Statistical Service (GSS), to quality assure our latest methods and estimates. Our methods are reviewed by:

• internal experts

• external experts such as the Methodological Assurance Review Panel (MARP) chaired by Sir Bernard Silverman with external members

• academics

• our Migration Statistics Expert Group 

These reviews provided useful feedback including how the use of admin data allowed the use of different definitions and being able to compare across those definitions, rather than only using the international standard as before. They also acknowledged the challenge of users becoming used to revisions and adjustments was a feature, rather than a limitation.

Understanding the strengths and limitations of each data source is an important part of how each additional data source affects the migration estimates in this transformed system. As part of the quality assurance of our new methods, we now routinely publish analysis of coherence of official estimates of migration against indicators provided by other sources. See Section 7: Consistency across data sources section in Long-term international migration, provisional: year ending June 2022.

User engagement and communication

Summary of OSR recommendations

The ONS should engage with a wide range of users so that their needs are well understood.

We have worked closely with our users to develop statistics that meet their needs, using a variety of forums to maximise our reach.

We continue to regularly meet with the UK Population Theme Advisory Board, the Migration Statistics Expert Group, the GSS Migration Statistics Steering Group and the Methodological Assurance Review Panel.

Statistical agencies representing the devolved nations - National Records Scotland (NRS), Northern Ireland Statistics and Research Agency (NISRA) and the Welsh Government - are integral members of our expert and steering groups and contribute regularly to our stakeholder groups. We also make sure our reach extends beyond the UK. We shared our research at the International Forum for Migration Statistics in Chile in January 2023 and this year will chair the UN migration expert taskforce on emigration statistics.

We reintroduced a cross-Whitehall group on migration analysis which meets monthly and has helped share and understand user needs across government.

We also meet bilaterally with many stakeholders to support policy decisions. We meet monthly with the Migration Advisory Committee, who hold responsibility for advising Home Office on the Shortage Occupation List and have worked very closely with HM Treasury to understand the impact our research has on fiscal policy.

We are also proactive in reaching wider audiences, for example, through the promotion of our research at various conferences and seminars. These include the Migration Statistics User Forum, and external conferences such as the British Society for Population Studies and the Royal Statistical Society. We hosted external forums to explain how we disaggregate migration statistics, attracting interest from many users including local authorities.

To keep our users regularly updated we have:

• introduced a monthly Population Statistics newsletter

• published two research reports in the past year

• published subscriber emails to promote and explain our work

We aim to reach all users and so have used the National Statistical website to publish topical blogs presenting our statistics in a user-friendly way. The statistics we produce are complex but have important messages. We published blogs in May and November 2022 putting the numbers into context, so they could be understood by a wide variety of users from the inquiring citizen to the most technical audiences. We also helped connect how our research supports the wider population statistics transformation as described in our International migration statistical design progress report: July 2022 article as well as through a broader set of blogs in 2022. We also use social media to reach a wider audience, using @ONS, @ONS_focus and our many tweeting statisticians on Twitter.

Engagement with users has enabled us to continually improve our statistics. For example, external expert advice during a census migration quality assurance exploration day provided us with the confidence to move away from the SSM towards an admin-based approach for estimates published in May 2022. Further, recent engagements with the Migration Expert Group helped assure us that revisions to estimates of non-EU emigration, published in November 2022, were justified and correct.

Users highlight the important role that population and migration statistics have in informing a wide range of decisions. This includes assessing demand for services nationally and locally (including education, housing and healthcare) and the plans for future UK immigration and labour market policy.

User feedback also shows clear support for our ambition. While our users welcome the move to put administrative data at the core of our statistics, they also highlight a wider range of needs which we are seeking to incorporate within our plans. These include the need for a broader range of estimates that encompasses the diversity of migration patterns and research into alternative definitions of migration to improve nuance in the statistics. The devolved nations, NRS, NISRA and the Welsh Government, are very much a part of our expert and strategic group and contribute regularly to our stakeholder groups.

We recognise there is still work to be done to engage new users with our statistics. We are exploring new opportunities to engage a wider audience through existing methods, and media channels, for example podcasts and webinars on specific topics such as measuring students.

We have made good progress, including providing break downs of immigration estimates by reason for migration. However, we are aware that our long-term international migration estimates do not yet cover all types of migration or fully reflect the reality of dynamic movement and mobility seen in today's population. 

There is strong interest from our users in receiving real-time estimates on the numbers of visitors and short-term migrants within the UK and the likelihood of these migrants staying long-term. In response, we are continuing to develop methods to meet the increasing need for flexibility in defining a migrant. Our International migration research, progress update: November 2022 builds on our previous analysis on supplementary estimates of migrants based on actual time spent in the UK, Exploring international migration concepts and definitions with Home Office administrative data (February 2020).

We are currently at the proof-of-concept stage, focusing on immigrants and using our existing administrative sources to identify evidence of:

• during and end of the month activity within the DWP's RAPID 

• travel dates recorded within Home Office Border Systems data and how we can use them to build up a picture of alternative lengths of stay and types of migration to the UK

Summary of OSR recommendation

The ONS should also improve the accessibility of migration data and information, so users are clear which is the latest data and who is responsible for producing it.

We recognize the complexity that users face in accessing and understanding statistics on migration published across the GSS. With many producers (including the ONS, the Home Office and DWP) producing estimates of similar, but often subtly different concepts, the risk of misinterpretation and misunderstanding is acute.

To address this, we are leading with partners from the GSS Migration Steering Group on a number of initiatives to improve coherence across producers. This includes the development of a GSS interactive dashboard which brings together all GSS migration statistics in one place. Further, we are working with our colleagues in the GSS Migration Steering Group to develop a "one-set of numbers" approach. The output of this work, which we expect to share publicly later this year, will enable users to understand how different GSS migration statistics interrelate with one another, and how best to use these statistics given specific needs.

The wider population transformation programme

Summary of OSR recommendations

The ONS should support users to understand the wider population transformation programme by using a consistent set of terms (recorded in a glossary) accompanied by a high-level roadmap.

Following the rebasing of the mid-year population estimates that are due for release in spring 2023, we aim to use all the available information to inform a revised data back series of long-term international migration flows over the course of the last decade, which we plan to publish in May 2023. Our ambition is to have coherent stocks and flows in future through the DPM. Estimates published covering the period year ending (YE) June 2020 to the YE June 2022, information from Census 2021, alongside available administrative data, provide the clearest possible picture of international migration flows over the last decade.

There will be a consultation on the future transformed system for population and migration statistics in summer 2023 and we welcome users views on transformed migration statistics. This will support the National Statistician's 2023 recommendation on the future of population statistics (PDF, 249 KB). In addition to the existing work on Statistical Population Datasets (SPD's) and the DPM already published there will be future research work published between now and June 2023.

Migration statistics are critical for the transformation of the Labour Force Survey and all population statistics. We have an ambitious programme of work with four transition states, occurring in May and November each of 2023 and 2024. Users can expect iterative progress on improved migration statistics in line with what is described in this article.

We are working with population statistics colleagues on publishing a workplan which will be made available on the ONS website. In addition, on 28 February 2023 we will share a blog with users on how our collective research is progressing, which will help users make informed decisions ahead of reaching out later this year to consult on the future of population and migration statistics.

The Office for Statistics Regulation's (OSR's) review on migration statistics encouraged us to improve and broaden our user engagement as well as ensuring that we have coherent plans across our transformation work. We recognise the need to continuously improve our methods with our users. If you would like to find out more or have any feedback, please email us at pop.info@ons.gov.uk.

In addition to exploring other data sources as they become available and existing sources as they are updated in the coming year we plan to:

• continue to engage with experts and users through our existing forums and meetings, additionally we will make the most of opportunities through upcoming conferences, events, and podcasts to reach a wider audience

• publish a Quality and Methodology Information report in May 2023 to explain to users the quality measures of our processes and data

• develop uncertainty measures around estimates, an important part of ensuring the appropriate use and interpretation of statistics; we are planning a working paper on the first iteration of our proposed uncertainty methods for May 2023

• use of an alternative data source to the International Passenger Survey (IPS) for the estimation of British nationals by November 2024

• research data on the use of new visa requirements for EU nationals, to allow us to estimate their migration patterns using a consistent method to non-EU nationals by November 2023

• report on how those with EU settled status are reflected in Home Office data and how we can use their travel information to estimate their long-term migration patterns by November 2023

• investigate how to account for irregular migration and asylum seekers in our headline estimates of long-term international migration by May 2023

• understand the migration patterns of students by developing a student emigration method by June 2024.

• assess the accuracy of machine learning model-based long-term international migration predictions against known outcomes at the individual record level, as well as their impact on the accuracy of provisional admin-based immigrant flows against the confirmed data back-series; we hope for initial results in May 2023