Levelling Up Subnational Data project, year 2 evaluation report, UK: April 2022 to March 2024

The Levelling Up Subnational Data (LUSD) project aims to improve the government's subnational data capabilities, while also providing policy makers and the public with evidence for the missions and metrics in the Levelling Up White Paper. The LUSD project is being delivered as a collaboration between the Office for National Statistics (ONS) and the Ministry of Housing, Communities and Local Government (MHCLG) (until recently known as the Department for Levelling Up, Housing and Communities (DLUHC)). This is a significant cross-cutting project for the period 1 April 2022 to 31 March 2025.

The project is aligned with the Government Statistical Service (GSS) subnational data strategy ambitions, which are to:

• produce more timely, granular, and harmonised subnational statistics

• build capability and capacity for subnational statistics and analysis

• improve the dissemination of subnational statistics

The overarching intended impact of the project by the end of year 3 is that policies and local decision making will be better informed by subnational data.

At the end of year 2 (year ending March 2024), substantial progress has been made towards the subnational data strategy ambitions.

More granular data have been made available, allowing users to build up flexible geographies for a wider range of local analysis needs. Development of more granular gross value added (GVA) was a particular achievement to fill an important gap in local evidence, allowing areas to understand economies at the most local levels. The project has also published more insights based on new data sources and methods, to enable a wider understanding of local issues. 

Development of the new data platforms reached an important milestone with the launch of the Beta service – Explore Local Statistics (ELS) – radically improving the way that users can find and compare statistics for their local area.

ONS Local became an established analytical service in all English regions in March 2023, and was fully operational across the UK by March 2024. The focus of the service has been on supporting local government by ensuring they have access to data and analysis to support evidence-based decision making. ONS Local also runs webinars and workshops to build capability across local government and the regional leads offer diverse support to local stakeholders, encouraging collaboration within and across regions. More recently, they have focused on understanding and filling local data gaps.

Overall, the ONS has published 22 outputs or products across the two years of the project (not including user-requested data from ONS Local), increasing the range of statistics produced for local areas, enabling better local decision making. See Section 9: Project achievements, for a summary of the work completed so far.

Main findings from the evaluation

The evaluation was based on four main evaluation questions, and this report is structured around them.

• Are local decisions better informed by new or improved granular data, analysis, and methods?

• Did new data platforms provide a valuable repository to find, visualise, compare, and download subnational data and statistics?

• Did ONS Local contribute to filling knowledge gaps by providing analytical support and advice?

• Are local decisions better informed by subnational data via ONS Local and new data platforms?

This evaluation report summarises the impact evidenced to date and what has been delivered by the end of year 2 (year ending March 2024).

Evidence suggests that:

• new or improved granular data, such as GVA, regional gross fixed capital formation (GFCF) and UK public transport availability, have enabled enhanced local-level analysis and decisions by local government

• new data platforms are a helpful resource to local government analysts and MHCLG in the early stages of analysis to quickly build a better understanding of a local area, feeding into decision making

• ONS Local is providing a valuable analytical service to local government and has contributed to local decision-making processes

• there are areas for improvement that are recommended for the final year of the project to strengthen the impact (see year 3 recommendations)

Year 3 recommendations

The following activities are recommended for the project's third and final year (April 2024 to March 2025). 

Across the whole project:

• increase awareness and reach to improve engagement and deliver more value and impact  

• improve data availability for devolved governments, where applicable, using different approaches such as alternative measures, to enable more informed comparisons between areas across the UK

• establish a sustainable model for continued support of local user-required services and outputs

For new data sources, methods and more granular local data:

• increase awareness and use of these data, seeking different ways to promote the data opportunities, including using the ONS Local network

• improve the timeliness of granular data and provide more guidance to increase users' understanding and use

• prioritise and deliver new and improved granular data, based on user needs gathered by ONS Local, ensuring this will provide the most impact for local and national users

For new data platforms:

• improve the functionalities to meet more user needs including adding more granular levels of data (such as county, combined authority)

For ONS Local:

• increase the reach of the analytical service from ONS Local (such as to academics and government departments) and establish a strategy to target those in local government who are less aware of the service

• continue to promote ONS Local and raise awareness, ensuring potential users are clear about its core services to support analysis and inform evidence-based decisions

• establish a more sustainable way to deliver priority local user needs across ONS and the GSS

Lastly, we recommend strengthening the monitoring and evaluation approach for the final evaluation by broadening the user groups involved in the activities. This includes targeting more central government stakeholders and decision makers, and developing a better understanding of the reach of the outputs.

The project is split into four largely distinct work packages which support the broader aims through delivery of specific services and outputs. There are expected outcomes and impacts resulting from the work packages (see Section 13: Evaluation methods).

ONS Local

An analytical advisory service for local leaders to support evidence-based decision making across local government, with dedicated analysts based in every region and nation across the UK.

New data platforms

A suite of public dashboards with openly available data – known collectively as Explore Subnational Statistics (ESS). These data insight platforms are intended for use by subnational stakeholders, including the public.

New data sources and methods

Publication of new data and methods made possible by expanded use of administrative data sources and use of innovative techniques, with the intention to improve the timeliness and granularity of insight available. 

More granular local statistics

Transformation of economic and social indicators by breaking down into small, more local, geographical areas. This service not only aims to increase the granularity of published subnational statistics, but also aims to improve the ability to compare subnational data across all parts of the UK.

The Office for National Statistics' (ONS's) Evaluation Strategy sets out the ONS's vision for embedding evaluation best practice across all work. To align with this, the Levelling Up Subnational Data (LUSD) project aims to publish evaluation updates throughout its lifecycle, increasing the transparency, value, and impact of the work conducted by the ONS.

The purpose of monitoring and evaluation for the LUSD project is to provide learning points throughout the project delivery and ensure continued accountability. Evaluating the success of the levelling up policy is not within the scope of this project.

This report summarises the impact evaluation and progress of the LUSD project at the end of year 2, covering the period April 2022 to March 2024.  

The evaluation is mainly based on data collected for evaluation purposes via a survey, interviews, and focus groups. See Section 11: Evaluation data, for more on the data sample and Section 13: Evaluation methods, for limitations to the evaluation approach.

Some services and projects are not fully evaluated here because they are either not completed or were published near the end of the reporting period. The project is funded until March 2025, after which a final evaluation will be published.

The evaluation approach is based on evaluating the high-level outcomes with four main evaluation questions – findings for these are summarised in Sections 4 to 7.

The project has produced several pieces of new or improved data and analysis (see Section 9: Project achievements). Evidence to date suggests that local government are producing enhanced local-level analysis from some of these data, indicating more evidence-based approaches to decision making, local interventions, strategy and policy. This includes new data on granular gross value added (GVA), regional gross fixed capital formation (GFCF) and UK public transport availability.

For some outputs published so far, there was not enough evidence to know if or how they had contributed to local decision making, and this needs to be monitored in the final year. Further analysis is required into the evidence gaps, and to find out whether the statistics are being well used for decision making, and if they are not, to understand the reasons.  

We asked the evaluation survey respondents the reasons they had used new or improved data. The most common reason was to explore data about an area.

For local government, half of the survey respondents had used these data for supporting decision making or informing policy (15 out of 24).

We asked, during interviews, which of these new or improved data had been used for decision making and how. Granular gross value added (GVA) was by far the most used (See Case Study 1).

Regional gross fixed capital formation (GFCF) had been used as a policy driver to promote investment into a Combined Authority region. Wider feedback to the GFCF team highlighted other examples that may have informed decisions, such as researchers and policy officers estimating regional capital stocks for UK and European countries.  

For the new data sources and methods, it was deemed too early to demonstrate evidence these were feeding into decision making. Most were in the early stages of development and outputs. One exception was the output on UK public transport availability, which was published in January 2023. There is some evidence of where this has been used; however more investigation of the impact on decision making is needed.

Clustering analysis, which groups UK local authorities with similar characteristics and outcomes, aims to inform local government with new information and enable collaboration across areas. To date, this work has not been widely used although we heard how it could be useful to interview participants in the future. Reasons that clustering analysis had not been used included lack of awareness, users needing more time, or using alternative data and methods that were already established with local users.

Case study 1: Gross value added (GVA) at local level has enabled more informed analysis and decision making

GVA is a standard measure of the economic activity taking place in an area. It comprises the majority of gross domestic product (GDP).

Methods were developed so that GVA can be broken down to the granular level of Lower Super Output Area (LSOA). The strength of data at this level is the ability to build up to flexible geographies of interest that are not constrained to pre-defined geographical boundaries.

Granular GVA was by far the most used and valued dataset from those interviewed, including participants from combined authorities, local councils, devolved governments, public bodies, think tanks, and the Ministry of Housing, Communities and Local Government (MHCLG). Interview participants said that the GVA data was enabling more informed local decision making and policies. They were using these statistics to inform business cases, strategy, planning and they were being used for evaluation.

One local government analyst worked in a large region with lots of variation at local level. Previously, GVA was at district level, which did not provide the level of detail needed. The analyst explained that being able to look at lower levels within districts helped to identify which areas were standing out in terms of economic growth and productivity, which fed into their local economic strategy.

Another local government user shared similar experiences. They described how the more granular GVA data has informed elements of research in terms of how they might attract future investment. Also, they made use of the data zone level granularity for Scotland to build up custom areas and then estimate how much university campuses are worth to the economy.

While overall the response to disaggregated GVA data was positive, there were suggestions for improvements. There was a preference to prioritise the timeliness of the data, rather than having more disaggregated data that would be out of date. A central government stakeholder described how the methodology and guidance could be improved to better compare data across the four UK nations.

The more granular GVA data have the most detailed insights to date, enabling more informed analysis and decision-making by different users.

Case study 2: Cross-project input has enabled data-driven policy making

The LUSD project supports the emerging needs for subnational work. A good example of this is the UK government's Long-Term Plan for Towns: data packs for 55 towns (published 28 March 2024 by MHCLG). These packs were full of local insights and intelligence to support the delivery of Town Boards, build capacity and understanding, and inform local residents.

Data on granular GVA and UK public transport availability were included in the packs. Visualisations from Explore Subnational Statistics (ESS) were also included, with ESS being used for initial exploration by MHCLG too.

The ONS team supported MHCLG in creating the first series of data packs, bringing together multiple datasets into easy-to-visualise and digestible information. This included development of automated processes, enabling efficient replication for more areas as required. ONS Local also provided regional intelligence as part of the review process.

The combined input from across the LUSD project has enabled data-driven policy making. The data packs have received positive feedback from users, including the town boards, local councils, senior leaders and ministers.

To what extent did new granular statistics fill gaps in evidence and user needs? 

It is expected that meeting user needs and filling gaps in local evidence will lead to better informed local decisions. An additional evaluation question was to consider the extent to which more granular data achieved this. This evaluation question was not a priority during the year 2 evaluation, but through analysing evidence the following observations were made on this topic.   

In evaluation interviews, local government users in a data-related role provided evidence that some gaps had been filled by providing lower-level data, allowing for more detailed analysis and understanding. Most examples of this can be seen using GVA (see Case Study 1).

The Ministry of Housing, Communities and Local Government's (MHCLG's) feedback was positive on the more granular data that had been produced so far. With use of new low-level data, MHCLG's understanding of local areas was enhanced and more informed (see case study 2).

In the evaluation survey, respondents from central and local government said how the more granular data had made a difference. For example, the data added to the evidence base at a local area, providing interesting topics to stimulate questions, and answering queries from local partners.

Others said they would like to see improved timeliness and reliability of the data, rather than producing it at a more granular, local level.

Improvements to the guidance for new granular statistics, including supporting documentation, was a common theme among users. Other improvements were centred around more geographical granularity (for example, county level) or alternative breakdowns and easier access to the data with improved communication when it is released. Also, clearer, and up-front explanation of geographical levels and coverage was advised.

UK data coverage has been provided for most of the new or improved data published. However, some improvement is needed in terms of data availability across all four UK nations that would help meet user needs and allow for better comparisons (see Case Study 3).

Evidence to date suggests that new data platforms – known collectively as Explore Subnational Statistics (ESS) – is providing a valuable resource for local government analysts and the Ministry of Housing, Communities and Local Government (MHCLG). It has helped conduct initial high-level analysis and build a better understanding of a local area at pace, ahead of more in-depth analysis to inform decisions.

We asked the evaluation survey respondents what they had used ESS for. The most common reason was finding data about an area. Over half had used the data platforms for visualising data about an area, looking at multiple topics and comparing multiple areas.

When asked what they can do now that they could not do before – interview participants said the new data platforms allow them to answer more questions, improve how they use the data and make comparisons to similar areas. 

Others were positive about how easy information was to find, compared with when these platforms were not in place. For example, having everything in one place and not having to search repeatedly through the ONS website.

Interview participants said they still use the source data or alternative data platforms (such as Nomis or internal data systems) alongside or in place of using ESS. Requests for future improvements to the geographic coverage of the data included in the platforms were a common theme, particularly for data relating to devolved governments (see Case Study 3). Also, there were several improvements to the platforms' functionality identified. For example, allowing more than four areas to be selected for comparisons and enabling identification of statistical neighbours.

Explore Local Statistics (ELS), the ESS Beta product, aims to have improved functionality and dissemination. It was launched on 26 March 2024 and therefore it was not evaluated during year 2. Evaluation in year 3 will look to explore the reach and scale of use of ELS, specifically the engagement from government analysts and the public.

Case study 3: Improvements needed to allow for more comparisons across Northern Ireland, Scotland and Wales

The new data platforms from ESS allow users to access and compare data at local levels across the UK. Comparisons with similar local areas is a common task for local and central government, to give a better understanding of how areas are doing and if any interventions are needed.

The availability of the indicators included in the data platforms is dependent on whether or how the data are collected. For example, education is a devolved topic area and therefore data from England, Northern Ireland, Scotland, and Wales may differ in terms of definition, measurement, and availability.

This has resulted in a disparity in what data are available on the new data platforms. Therefore, users of data on Northern Ireland, Scotland, and Wales sometimes had difficulties making meaningful comparisons between local areas. This could result in missed opportunities to fill knowledge gaps and inform decisions.

Feedback from users based in those countries was to ensure data is on an equal availability to England in future. A suggestion was to explore the feasibility of using proxy indicators with caveats where data are not available, as a perfect comparison was not essential for their needs. This is a recommendation for year 3 of the project.

Much of the activity of ONS Local in year 2 was centred around establishing and maturing the service offering and understanding user need and local data gaps.

Evidence at the end of year 2 suggests that ONS Local is providing a valuable analytical service to local government who are aware of it and need the support. This has helped provide new insights, and analysts have spent less time and resource looking for local data. However, not all local government users were aware of it or needed this analytical service from ONS Local, such as those with sufficient analytical resource and capabilities.

In the evaluation survey, we asked respondents which ONS Local services they had accessed. The most common reason selected was attending a webinar or workshop. However, this may be biased by these events being used to advertise the evaluation survey.

Over a third had used ONS Local to support analytical queries. Around one in five said ONS Local provided more extensive support with analytical projects, which are larger in scale and often collaborative.

The evaluation survey highlighted positive themes on the value of ONS Local to support local analysis. These included links into the Office for National Statistics (ONS), access to ONS expertise, and support to navigate ONS data.

Alongside supporting analytical queries, ONS Local provides analysis and product creation, such as user-requested data. In follow up interviews, those who received this service were positive about the experience and support received (see Case Studies 4 and 5).

Some interview participants were yet to use the ONS Local analytical service, particularly those with more analytical resource. Reasons given included using in-house analysis, only basic analysis required so did not need ONS Local, and established relationships with ONS output teams. For local government, the need for analytical support was greater for those with less resource and capability.

However, a lack of awareness of ONS Local and the wider project was highlighted (see Section 8: Other findings and Case Study 6).

Case study 4: ONS Local analytical support resolved data issues and shared learning

ONS Local provide support with analytical queries and projects, either through regular meetings with a regional analyst or as and when required.

A local government analyst found an issue with classifying qualifications data needed for a skills strategy. Speaking with other local governments, they found discrepancies using the same data. Therefore, the analyst asked for help from the ONS Local lead for their region, who helped resolve the data issues and discrepancies.

Using the ONS Local network of analysts in each region, they were also able to share the issue and resolution with analysts across local government.

ONS Local's support ensured quick resolution of the issue, so users could continue their analysis to fill knowledge gaps. Using the established ONS Local network and communication channels meant that dissemination was quick and easy, leading to wider impact and ensuring others were also gaining more valuable insights from the data.

Did ONS Local contribute to improvements in local analysis capabilities/capacity?​

Building capability is part of the analytical service from ONS Local. Understanding the contribution from ONS Local in supporting capability and capacity of local analysis was not a priority evaluation question for the year 2 evaluation, however the following evidence was found on the topic.

ONS Local hosts webinars and workshops aimed at training civil servants and local analysts how to better use data. There has been positive feedback on these events, with evidence suggesting they are adding to the skills of local government analysts – from those developing their data skills to those who are more experienced but needing to navigate the data landscape.

The PowerBI workshops were a series of sessions to introduce and explore the functionality of dashboarding tools. These were viewed as successful, with around 600 people attending on average. Positive feedback was collected from those in local government who were interviewed. They liked that the workshops used data and examples that were relevant to them. Almost all said they were likely to use the tools in their own work.

Evidence at the end of year 2 suggests that ONS Local and new data platforms have fed into decision-making processes in local government.

Evaluation feedback showed that ONS Local are supporting improved local decision making through various ways, including:

• by facilitating skills sharing

• supporting the finding and accessing of data

• offering guidance on data use

• providing user-requested data

The wide-ranging support ONS Local provides to local government more generally was evident throughout analysis from multiple sources.

The extent to which ONS Local is contributing to improved decision making will be explored further in the final evaluation.

For new data platforms, 34% of the evaluation survey respondents who used the platforms used them for supporting decision making, and 23% had used the platforms to inform policy development (Figure 2). Whereas, in the follow-up interviews, limited evidence was provided to directly link these resources with decision making.

Interviews with the Ministry of Housing, Communities and Local Government (MHCLG) highlighted the use of new data platforms at the start of gathering evidence for decision making and evaluating levelling up work (for example, the Long-Term Plan for Towns data packs, see Case Study 2).

Case study 5: Employment data made available to better inform local decisions

ONS Local provides analytical support including producing user-requested data and analysis where required.

Data on graduates employed in non-graduate roles were previously published by the ONS. An economic researcher from a public body wanted to use this to support evidence gathering on the quality of jobs at a local level. However, the existing data did not cover their local area. Therefore, they asked ONS Local to reproduce the dataset.

ONS Local saw there was a wider user need and therefore expanded the data to cover more local areas. Once published it was highlighted in the ONS Local newsletter for awareness across the local analyst network.

Local decisions were better informed because of ONS Local's actions. The researcher who made the original request could explore discrepancies between graduate skills and job roles and add to their evidence. Another local government user, who found out about the data from the newsletter, used it as part of a toolkit for investment decisions made within the local authority.

A common theme across the feedback to date has been the lack of, or limited, awareness of the Levelling Up Subnational Data (LUSD) project overall. Understanding this issue will help assess whether the project reached its target group.

The wider impacts of the project are expected to reach various local user and stakeholder groups, including central and local government, research community and the public. However, it is hard to gauge the impact on the public so far.

The evaluation survey showed that 23% of respondents had not used the project (see Table 1 in Section 11: Evaluation data). The main reason respondents gave for not using the project was that they were unaware it existed. 

We explored this further in the follow-up interviews to understand if there were any barriers or ideas to improve awareness.

Common feedback were issues with finding the information on the ONS website, which was consistent with findings from the ONS annual stakeholder satisfaction survey. Interview participants said they use newsletters to help navigate and keep up to date, however this was time intensive for them. ONS Local regional leads were also used to help find data and keep updated.

There was also recognition from participants that there is a lot of information available across the wider data landscape, which might be affecting the project's awareness and use.

Case study 6: Improvements needed for a greater awareness of LUSD and specifically ONS Local services

Lack of awareness of ONS Local (and the wider project) was a commonly observed theme throughout the evaluation at the end of year 2.

Out of the 144 evaluation survey respondents, 33 had not used any of the project's services or outputs. A further 40 respondents had not used ONS Local, with just under half (18 of the 40) of those being unaware it existed and just under a third (13 of the 40) who said they did not need to use it. The evaluation survey was promoted at ONS Local events, as well as with existing stakeholders and working groups of LUSD. ONS Local webinars and workshops were attended by a wide-ranging audience, including people who may not have been aware of, or familiar with, the service before attending. See Section 11: Evaluation data, for more information on the evaluation sample.

The ONS annual stakeholder satisfaction survey (December 2023 to January 2024) showed further lack of awareness of ONS Local. Only 35% of local government respondents had heard of ONS Local (15 out of 43), and this was even fewer for central government respondents (7 out of 26).

During the evaluation interviews, we spoke to users about their awareness of ONS Local in more detail and how they more generally find out about and navigate the data landscape. Some had used the ONS Local service without realising it. Others were not aware of the analytical support function that ONS Local offers.

One local government researcher based in Wales explained they were aware of the initiatives but had forgotten about it. They had attended an ONS Local meeting but were not aware of the user-requested data that were available or that they could request their own bespoke analysis. When they wanted to find data, their first port of call was to visit the Stats Wales website and they found Nomis a useful source.

As a result of this evidence, increasing the reach and improving the awareness of ONS Local are recommendations for the final year of the project.

ONS Local

The ONS Local service went live in the English regions in March 2023 (end of year 1). Bespoke services in Wales, Scotland and Northern Ireland were developed in partnership with each of the devolved governments over the following year. The ONS Local service became fully operational across the UK by March 2024 (end of year 2), with continued engagement and promotion to users.

To inform users on the progress of the service, two blogs were published in the past year, alongside monthly newsletters:

• our Unlocking the potential of local data blog post

• the Harnessing the power of local statistics with ONS blog post (gov.uk)

• the ONS Local monthly newsletters

In year 2 of the project (April 2023 to March 2024), over 300 people from 176 organisations requested ONS Local support. This includes providing analytical advice and producing analysis where needed. Examples of responses to user-requested data include:

• our ONS Local: Employed graduates in non-graduate roles in parts of the UK, 2021 to 2022 dataset (published on 31 August 2023)

• the Broadband coverage across Norfolk map (arcgis.com) (published in November 2023)

• our ONS Local: Business Ownership by Ethnicity Group in Liverpool City Region, 2017 to 2022 dataset (published on 24 January 2024)

• our ONS Local: Public houses and bars, London MSOAs, 2001 to 2023 dataset (published on 1 March 2024)

A full list of requested data published by ONS Local can be found on the ONS Local page.

ONS Local has hosted around 50 events, webinars and workshops in the first two years of the project with, on average, more than 400 people attending each workshop, and just under 100 attending each webinar (you can find details of all ONS Local events on Eventbrite). These have helped connect users with the developments happening in subnational statistics, to build capability and to share learning. Over 3,000 people are signed up to the ONS Local events mailing list.

New data platforms

Explore Subnational Statistics

The Explore Subnational Statistics (ESS) service has been introduced iteratively.

On 26 March 2024, our Explore Local Statistics (ELS) service was launched (ESS Beta). ELS is a digital dissemination service allowing users to find, compare and visualise statistics about places in the United Kingdom. At the time of its launch, the service hosted 57 different indicators, such as employment, school attendance and life expectancy. This is refreshed on an ongoing basis with new or updated datasets.

Before the launch of the ESS Beta, the following two services were in place:

• our Subnational Indicators Explorer web page – the ESS Alpha prototype – an interactive tool to compare a local authority and the UK average (median) local authority by different indicators, launched on 2 February 2022 and updated quarterly with the last update on 21 March 2024 before it was discontinued following the launch of ELS

• our “Find facts and figures about areas in the United Kingdom” web page – published in March 2023, now part of the ELS user journey

Additional products

The following ONS products were supported by the ESS team.

For Census 2021:

• our Census Maps web page – maps to find out what people's lives were like across England and Wales in March 2021 (published on 2 November 2022)

• our Build a custom area profile web page – create your own profile for local areas in England and Wales using Census 2021 data (published on 17 January 2023)

Other topic-based, semi-automated bulletins:

• our Employment, unemployment and related statistics for your area article – labour market indicators at local and unitary authority level for Great Britain (published on 5 October 2023 and regularly updated, in collaboration with the labour market team)

• our Housing prices in your area article – house prices and private rental prices for local authority areas across Great Britain (published on 20 March 2024 outside of the LUSD project but repurposing the code and expertise of the above labour market article)

New data sources and methods

The following new data and methods were published by the end of year 2.

Clustering analysis that groups UK local authorities with similar characteristics and outcomes comprises:

• initial analysis to inform local users how they could identify similar local authorities in England using clustering methods (published on 24 February 2023) in our Clustering local authorities against subnational indicators, England article

• our dataset expanded to UK coverage for the clustering similar local authorities in the UK (published on 23 February 2024) – which was then implemented as part of the compare function of the ELS service

• our methodology article on the clustering similar local authorities in the UK – more information on methods used in the clustering analysis (published on 23 February 2024)

Consumer card spending analytical series comprises:

• our Regional consumer card spending, UK article – covering overall consumer spending habits (published on 6 November 2023)

• our Consumer card spending, flow of spending across the UK article – analysis of consumer card spending trends covering where UK cardholders are spending money, on a local level (published on 25 March 2024)

Other publications:

• our Using open data to understand hyperlocal differences in UK public transport availability article (published on 20 January 2023 by Data Science Campus)

• our Access to sports facilities, supermarkets and museums at local level article (published on 7 March 2024)

More granular local statistics

The project has so far developed statistics at granular levels, presented in the following publications.

Disaggregating UK annual gross value added (GVA) publications:

• our Introducing GVA at lower levels of geography article (published on 24 January 2023)

• our Breaking down GVA to lower levels of geography (Lower-layer Super Output Area (LSOA), Data Zone (DZ) and Super Output Area (SOA) levels) article (published on 31 January 2024)

Research and development statistics publications:

• our Regional UK business research and development methodology (published on 17 April 2023)

• our UK public-funded gross regional capital and non-capital expenditure on research and development article (published on 17 April 2023)

Public sector finance statistics publications:

• our Using administrative data to improve public sector finance statistics, UK article (published on 29 June 2023)

• our Using local authority financial data to improve the granularity of public sector expenditure, UK article (published on 29 June 2023)

Well-being publications (well-being is one of the levelling up missions):

• our Review of the UK measures of national well-being, October 2022 to March 2023 article (published on 5 July 2023) – as a result of the review, subnational breakdowns, where possible, were included in the accompanying datasets to the UK measures of national well-being dashboard

• our Threshold estimates of personal well-being from the Annual Population Survey, by local authorities in the UK, January 2020 to December 2022 dataset (published on 14 December 2023)

Other publications:

• our Experimental methodology for producing UK interregional trade estimates (published on 28 July 2023)

• our Experimental regional gross fixed capital formation (GFCF) estimates by asset type dataset (published 8 December 2023)

• our Disaggregating UK subnational gross disposable household income to lower levels of geography: 2002 to 2021 (published 26 March 2024)

UK-wide data

The Office for National Statistics (ONS), the devolved governments, and the Ministry of Housing, Communities and Local Government (MHCLG) have been working together to join up data across the UK by creating new UK-wide data and analysis, as explained in this article on the Government Analysis Function website. For subnational or local data, this allows for more indicators to monitor policies and understand regional disparities.

In general, UK coverage was provided for the new or improved data published from the LUSD project to date. This has been achieved by developing methodologies to bring together data from the four nations, often overcoming challenges, some of which were longstanding.

Additional data and analytical support for local insights

The LUSD project supports the emerging needs for subnational work, alongside the work outlined in the theory of change (ToC) and described in the sections above. The ONS works closely with MHCLG to provide additional data and analytical capacity to support place-based insights and decision-making, for example:

• the Levelling Up Partnerships methodology note (gov.uk) – the LUSD project contributed subnational data that were used to inform data visuals, text and tables, also providing peer-review of data packs on the Levelling Up Partnerships and created an automation process for producing the packs (this collaborative work is ongoing)

• the Long-Term Plan for Towns: data packs for 55 towns (gov.uk) (published on 28 March 2024). The ONS team supported MHCLG with the creation and automation of the first series of the data packs, enabling replication for other areas – see Case Study 2 (this collaborative work is ongoing)

Planning and scope for the Levelling Up Subnational Data (LUSD) project was set by the Office for National Statistics (ONS) and the Ministry of Housing, Communities and Local Government (MHCLG) in early 2022.

Originally a cross-government collaboration platform was planned. The ambition was to have a full suite of dashboards to support decision making and analysis for central government in a coherent and consistent way, powered by the Integrated Data Service (IDS). This plan was changed at the end of 2023 because of functionality needs and the changing priorities of the project. Cabinet Office has the platform and technical infrastructure to support the development of cross-government dashboards (and secure sharing), and the IDS will be used for any secure data research. For the LUSD project, development has been focused on the data pipelines for the Explore Subnational Statistics (ESS) data platforms, which also support the emerging user needs, such as the Levelling Up Partnerships (LUPs).

Review of the original theory of change (ToC) happened at the mid-point of the project, to ensure this reflected the current status and future plans. Other changes were sensible adaptations as opportunities or challenges were found. For example, the maturing of the ONS Local service has meant some of the outcomes that were initially agreed have been updated to reflect the development of the service.

Baseline and year 1 (April 2022 to March 2023)

The data for the baseline were collected in autumn 2022 by combining desk research, meetings with stakeholders, and a benefits workshop, to better understand the current state and gain an understanding of problem areas the project will seek to address. Meetings were held with stakeholders from across the Office for National Statistics (ONS) and the Ministry of Housing, Communities and Local Government (MHCLG), as well as local stakeholders across UK regions.

In September 2023, our Levelling Up Subnational Data project monitoring update for year 1 was published, which provides a summary of the status at baseline and year 1.

Year 2 (April 2023 to March 2024)

An evaluation survey was open between January 2024 and March 2024 for users and stakeholders to provide views and experiences of using the project's services and outputs. Questions included how, if at all, these have contributed to decision making and any improvements they would like to see. The survey was anonymous, with the option to opt into a follow-up interview.

Interviews were conducted with those who agreed to participate. The interviews gathered in-depth qualitative information about how the ONS project and service contributed to decision making, improvements in the government's subnational data capabilities and wider understanding of regional disparities. In addition, interviews and focus groups were held with MHCLG colleagues.

To evaluate the delivery process, focus groups were held with ONS project teams and contributors to gather ideas and group opinions. 

Data from the survey, interviews and focus groups were analysed using:

• content analysis to reduce the unstructured text into manageable data relevant to the evaluation questions

• thematic coding to identify data linked by common topics, providing a list of indexed categories

Case studies included in this report were based on the experiences of users during interviews and focus groups.

Other data were sourced to complement the evaluation data collected, including:

• pre-post data from polls at the start and end of ONS Local webinars and workshops to assess any change in awareness or capability and future intentions

• management information on the use of publications on the ONS website 

• our ONS annual stakeholder satisfaction survey, which was open between 6 December 2023 and 22 January 2024, and included a section on ONS Local

Evaluation data sample

A total of 144 responses to the evaluation survey were received after it was promoted to existing stakeholders, collaborators and working groups. It was also shared at ONS Local webinars and workshops, which were attended by a wide-ranging, sometimes very large, audience including people who may not have been aware of the project or ONS Local before attending.

Background information on the survey respondents showed:

• 56 were in data-related job roles (such as statisticians, analysts, researchers, data scientists, economists) and 5 were in other professions (83 were in unknown roles)

• 52 were from local government, followed by 11 from national government departments (including arm's-length bodies) and 10 from devolved governments (59 were from unknown organisations)

• a broad spread of subnational interest across the UK regions and nations, with 94 interested in only one geographic area (1 respondent's area(s) of interest were unknown)

Survey respondents were given information on the project's services and outputs. Most said they used some part of the project. Around one in five had not. This was similar when considering respondents just from local government.

Fifteen interviews with survey respondents were held, to gain a more in-depth understanding of their use. The sample ranged from those who have not used the project at all to those who have used every part in detail.

This group comprised of eight interview participants interested in data within England. Two were interested in data within Wales and a further two in Scotland. One participant was from Northern Ireland.

In terms of organisation types represented, the sample included seven participants from local government. Other organisation types represented included public bodies, central government departments, pan-regional partnerships, think tanks, and charities.

A further six interviews or focus groups were held with MHCLG colleagues, most of whom work closely with the project.

Subnational

The term "subnational" refers to all data that are produced for the 12 international territorial level 1 (ITL1) areas in the UK and smaller geographical areas.

International Territorial Levels (ITLs)

As of 1 January 2021, the internationally comparable regional geography for the UK is the International Territorial Levels (ITLs) geography. This has replaced the nomenclature of territorial units for statistics (NUTS) geographies for the UK that were operational when the UK was a member of the European Union. The International Territorial Levels and associated lookups are available to download from our Open Geography portal.

Devolved governments

A collective term for the executive bodies in Northern Ireland, Scotland and Wales: the Northern Ireland Executive, the Scottish Government and the Welsh Government.

For some topic areas, such as where policy is devolved, each country may have several bodies responsible for producing statistics. This can make finding data on a particular topic difficult. Our Statistics across the UK web page provides more support and guidance on the matter.

Theory of change (ToC)

A theory of change (ToC) for the Levelling Up Subnational Data (LUSD) project was created in summer 2022. To develop this, subnational statistical needs of local decision makers and the public were defined and translated into intended outputs, outcomes, and impacts, mapped against the inputs and activities required to achieve them. The LUSD ToC sets out how the LUSD project can achieve several outcomes. This was reviewed and updated prior to year 2 evaluation (October 2023 to January 2024).

The overarching impact of the LUSD outcomes is that policies and local decision making are better informed by subnational data. This aligns with the following impacts and outcomes from the Office for National Statistics' organisational theory of change:

• ONS impact – increased proportion of decision-making and debate are informed by ONS statistics

• ONS outcome – top-quality published statistics on prices, GDP, and employment

• ONS outcome – more responsive approach to user needs and priority issues

• ONS outcome – increased accessibility of outputs resulting in more public engagement

Outcomes and impacts 

For ONS Local, the high-level outcomes are:

• improved capability and capacity in subnational statistics and analysis for analysts in local government

• local government spend less time and resource looking for subnational data, and have increased analysis support, to inform their decision making

• increased transfer of insights across local and central government

For new data platforms, the high-level outcomes are:

• local government spend less time and resource looking for, and drawing insights from, subnational data to inform their decision making

• subnational statistics are easier to find, more accessible and better presented, altogether reducing the burden for users

For both new data sources and methods, and more granular local statistics, the high-level outcomes are:

• better-informed central and local decision making and policies, with improved transparency and rationale of place-based spending

• data, methods, and insight available at a wider range of subnational geographies to better inform central and local government decision making

The wider impacts to different local user and stakeholder groups are:

• local government – evidence-based decisions for local decision makers

• central government – evidence-based policy interventions that are appropriate to places and better decide where and how to allocate spending

• both local and central government – evidence-based evaluation of levelling up missions and policies

• research community – ability to examine and understand the drivers and determinants of local growth and the policies necessary to drive this growth

• public – improved engagement and understanding, combined with greater transparency of local-level statistics, allowing the general public to understand outcomes in their area and to hold local leaders to account for those outcomes

Methods

Impact and process evaluation are used for this project, using an iterative approach over the years. For example, this current year 2 evaluation will build on the approach, questions, methods, and evidence gathered in year 1 (April 2022 to March 2023). 

A range of monitoring and evaluation methods were considered to measure the impact of the LUSD services and outputs, in line with HM Treasury evaluation guidance, known as The Magenta Book.

Theory-based evaluation (TBE) methods were used alongside quantitative and qualitative research methods. TBE methods will be used for both impact and process evaluation as described in Section A1 of the Magenta Book Annex A: Analytical methods for use within an evaluation (PDF, 495KB).

For this impact evaluation at the end of year 2, the principles of realist evaluation were applied to the main evaluation questions considering different user groups. Context-mechanism-outcome (CMO) statements were developed during the analysis.

For year 3 and final evaluation, we will build on these methods by testing the CMO statements, considering if and how they apply to different user groups in different contexts. In addition, we will apply contribution analysis to further explore the wider context.  

The approach and methods were based on discussion with, and advice from, the Evaluation Trial and Advice Panel (ETAP) in April 2024.

Limitations of the evaluation at year 2

The findings reported here are limited by the following aspects of the evaluation.

The sample, that the evaluation findings were based on, was a small section of the users or stakeholders with a relatively small number of interviews. Naturally the sample will include the most engaged and most data savvy – those less engaged in the project will be less represented. In addition, the different views of central government are not fully represented. In year 3 we will use different approaches to reach a wider user group and use more "blind" questioning to minimise attribution bias, as respondents' awareness of the project might be low.

The timing of the evaluation at the end of year 2 covers the period April 2022 to March 2024. Some services and projects were not fully evaluated because they are either not completed or were published near the end of the reporting period.

More time is needed for people to use the data and services and to understand any impact on decision making. In year 3, this will be addressed by gathering evidence directly from decision makers to gain a better understanding of the contribution that the services and statistics might have had to decision making.

The Levelling Up Subnational Data (LUSD) project is complex, with many different services and outputs across the ONS website. Combining this with a lack of awareness meant that identifying what respondents have used from the project was challenging, especially for the new data platforms and the new and improved granular data. Therefore, this has not been fully attributed in the evaluation to date.

During interviews and within the survey, respondents were asked questions such as what difference the project had made and what they can do now that they could not do before, if anything. Responses to these questions could be affected by recall bias (not accurately remembering and reporting past experiences).

The project is funded until March 2025, after which a final evaluation will be published.

For year 3 and final evaluation, we will build on the use of the approach and methods applied in year 2. This includes testing out the context-mechanism-outcome (CMO) statements identified. We will further consider the effect of the alternate theories identified so far (other means outside of the project that may have resulted in the same outcome).

To minimise some of the limitations, we will consider evaluating the contribution from the project overall. Evaluating the contribution to decision making is where this could be particularly important, where people often draw on multiple sources and services to inform decisions.