Transformation of UK consumer price statistics, groceries scanner data analysis: April 2025

• We have produced indicative estimates of the impact of introducing groceries scanner data into UK consumer price statistics; estimates are subject to change following further quality assurance.

• The average indicative change to the Consumer Prices Index (CPI) annual rate from the introduction of groceries scanner data between January 2019 and June 2024 was negative 0.04 percentage points; this already adjusts for the changes we introduced in February 2025.

• Impacts are larger for more granular indices, where the relative proportion of groceries scanner data is higher.

• Estimates will not be included in our inflation statistics this year, as we reported in January; we plan to parallel run groceries scanner data for a year before incorporating them into live production in 2026.

• Because of the complexity of data and methods and the importance of these statistics, further quality assurance is required to ensure they are of the highest quality before integrating into our consumer price statistics.

• We will not be revising our consumer price statistics following the introduction of scanner data in 2026.

• These methods changes will be introduced into our Consumer Prices Index including owner occupiers' housing costs (CPIH), Consumer Prices Index (CPI), Retail Prices Index (RPI) (subject to the process described in Section 21 of the Statistics and Registration Service Act 2007), and Household Costs Indices (HCIs) headline measures in March 2026; we will not discuss HCI changes in this article.

We are currently undertaking an ambitious programme of transformation of our consumer price statistics. This is to safeguard the production and publication of official statistics that serve the public good, as stated in the Code of Practice for Statistics. We explain this programme further in our Transformation of consumer price statistics: August 2024 article. This includes:

• identifying new data sources

• improving methods

• developing systems

• establishing new processes

We have been delivering these improvements over several years, reflecting the complexity and intricacy of what we are aiming to achieve.

We introduced improved imputation methods, consumption segments, and Northern Ireland private rental price statistics into our UK consumer price statistics from February 2025 (published March 2025). We also moved our consumer price statistics production onto a new sustainable cloud-based system from February 2025. The move to a cloud-based platform and the development of new methods will allow us to use admin data at pace in the future.

The next improvement we intend to make is introducing groceries scanner data in February 2026 (published in March 2026), as described in our Introducing grocery scanner data into consumer price statistics article. We will introduce scanner data for 50% of the groceries market, with the remainder still covered by the existing in-store collection. We are already able to produce consumer price inflation including groceries scanner data. However, because of the complexity of data and methods and the importance of these statistics, further quality assurance is required to ensure they are of the highest quality.

We plan to parallel run groceries scanner data for a year before incorporating into live production in February 2026 (published in March 2026). This is because methods and data changes must be made alongside the basket and weights update in February each year. Although we will not publish these statistics regularly this year, an updated impact analysis will be released prior to their incorporation into live production in 2026.

Scanner data refers to data that are collected by retailers at the point of sale. These are aggregated by product and do not identify individual consumer purchases.

There are many benefits to using scanner data, compared with our current sources, including:

• improved product coverage

• accounting for price variation within a month

• improved geographic coverage

• inclusion of more discount types

• expenditure per product

In traditional field collection, our price collectors collect price quotes once per month, or twice for certain volatile product categories. These are collected from a sample of 141 UK locations. We receive scanner data covering all transactions and all stores, which are typically aggregated daily or weekly. We plan to use data covering a three-week period to calculate an average price per product. This will allow us to better reflect the various prices a product is sold at throughout the month.

We also limit our traditional price collection to a sample of representative products. However, we have much broader coverage of products from scanner data. For example, we collect price quotes for carrots to represent a range of vegetables in the traditional collection, whereas for the same category in the scanner data we also include prices for parsnips, beetroot, ginger, radishes, celeriac, horseradish, swedes, and fennel. We also include nearly all products within the scope of the category, rather than a targeted sample of those products judged to be most popular, as in the traditional collection.

Scanner data also provide transaction prices, rather than advertised prices. This means we can observe the average price consumers have actually paid for each product, including discounts not currently included like multibuy promotions and loyalty scheme discounts.

A further benefit of scanner data is that they contain details of how many of each product were sold. We have always had weight information for aggregating above the item level. This new information enables us to introduce weights when aggregating products at the elementary aggregate level. This allows us to weight individual products according to their economic importance. For example, if consumers are spending more on Pink Lady apples than Braeburn apples, then Pink Lady apples will have a greater influence on the inflation rate for apples.

Scanner data are of a much larger scale and have different features, compared with our traditional data. Because of this, we have developed new methods and systems to process scanner data and integrate it into our consumer price statistics. For example, we will adopt multilateral index methods for the groceries scanner data. This is consistent with our approach for introducing alternative data sources for rail fares in 2023 and second-hand cars in 2024. These methods allow us to make better use of these data sources and to account for dynamic changes in the market and consumer spending habits.

More information on the methods developed for use with groceries scanner data can be found in our Introducing grocery scanner data into consumer price statistics article.

We present the differences between our series incorporating groceries scanner data and our series using the improved methods and consumption segments we introduced in all our published consumer price inflation measures in February 2025 (published in March 2025). For the series including groceries scanner data, we use a combination of scanner data where available, and data from our traditional price collection for the remainder of groceries. We compare this with consumer price statistics including the methods improvement made in February's index, rather than the published series. This allows us to understand the impact of scanner data.

For more information on the series presented using new methods we introduced in February 2025, please see our Impact analysis on transformation of UK consumer price statistics: January 2025 article.

There could be various reasons for differences between indices with and without scanner data. We describe potential reasons for this in this section. We will also investigate this in greater detail in the coming months.

Different product coverage

Scanner data cover more types of products. This is because consumption segment definitions are broader than traditional item definitions. This is also because nearly all products with expenditure, for a given consumption segment, are represented in scanner data. This is in contrast with the traditional collection, where price collectors are instructed to choose products that are likely to be the most popular. For example, they may use shelf space as a proxy for popularity, use their own market knowledge, or speak to retailers.

Different treatment of promotions

There are more types of promotions, like multibuy and loyalty scheme discounts, included in scanner data than in traditionally collected data. We can account for the uptake of these offers accurately with scanner data, because we have exact sales revenues per product. In the traditional collection, price collectors only collect shelf prices that are not conditional on other factors, like membership of loyalty schemes or purchases of additional products. However, including more product discounts does not inherently make indices lower. This is because price indices measure price change over time, not absolute price levels.

Different temporal and spatial coverage

Prices are derived from data covering three weeks in scanner data. The traditional collection is from a single point in time, or two points for certain volatile product categories. This means that the average monthly price derived from scanner data reflects the various prices a product is sold at throughout the month. Scanner data also has wider spatial coverage, so they can capture prices across all of a retailer's stores within a region, rather than just a few stores.

Different index methodologies

The methods we use for groceries scanner data are multilateral index methods, as described in our Introducing multilateral index methods into consumer price statistics methodology. These methods use product weights to account for the economic importance of each product and to account for products entering and leaving the market.

Price quotes within items are weighted by shop type and/or region with our traditional index methodologies. However, weights are not available at the micro price-quote level.

Figure 1 shows the indicative impact of introducing groceries scanner data for Consumer Price Index (CPI).

Table 1 shows the indicative impact of introducing groceries scanner data on CPI annual and monthly rates of change, per year, rounded to two decimal places.

Figure 1 shows that the annual rate of change for CPI was 0.04 percentage points lower on average, when introducing groceries scanner data. However, the impact of groceries scanner data changes over time.

The inclusion of groceries scanner data has a positive impact on the annual rate for CPI for most months between March 2020 and November 2021, during the coronavirus (COVID-19) pandemic. The better representation of product discounts in scanner data, compared with our traditional collection, is one factor that could be contributing to this.

Research by the Institute for Fiscal Studies shows that a reduction in promotions was a notable cause of inflation in the early stages of the pandemic, as described in their Grocery prices and promotions during the COVID-19 pandemic briefing note (PDF, 1.6MB). Scanner data accounts for multibuy discounts and many loyalty scheme-related discounts and the traditional collection does not. So, any reduction in the types of discounts available would only be reflected in the scanner data and could be contributing to higher annual growth rates.

The introduction of groceries scanner data had a negative impact on the annual rate of CPI from December 2021 to June 2024. This was a period of high inflation. One possible reason why the inclusion of scanner data may have had a negative impact is that these indices are more representative of actual consumer expenditure. Consumers may have bought less inflationary goods, which is better accounted for by the GEKS-Törnqvist multilateral method. Further information on the causes of the differences can be found in Section 5: Causes of differences associated with introducing grocery scanner data. We will investigate this in greater detail in the coming months.

Figure 2 shows the indicative impact of introducing groceries scanner data for the Consumer Prices Index including owner occupiers' housing costs (CPIH).

Table 2 shows the indicative impact of introducing groceries scanner data on CPIH annual and monthly rates of change, per year, rounded to two decimal places.

The annual rate of change of CPIH was 0.03 percentage points lower on average, when introducing groceries scanner data. CPIH follows a similar trend to CPI with slightly smaller differences, because of the slightly lower weight the "food and non-alcoholic beverages" and "alcoholic beverages and tobacco" categories have in the CPIH compared to the CPI. For more information on the consumer prices weights, please see our Consumer price inflation, updating weights: 2025 article.

Figure 3 shows the indicative impact of introducing groceries scanner data for the Retail Prices Index (RPI).

Table 3 shows the indicative impact of introducing groceries scanner data on RPI annual and monthly rates of change, per year, rounded to two decimal places.

The annual rate of change of RPI was 0.03 percentage points lower on average, when introducing groceries scanner data. The average annual growth differences per year for RPI were similar to CPI and CPIH in 2020, 2023, and 2024, but larger in 2021 and 2022. Differences between RPI and CPI/CPIH were partly caused by imputation differences during the pandemic relating to the different classification systems used for RPI and CPI/CPIH. The "alcoholic beverages and tobacco" division has a higher weight in RPI, compared with CPI and CPIH. This is because of the inclusion of licensed alcohol sales within this division for RPI, which contributes to the differences seen.

Price movements for higher-weighted component indices will have a greater impact on the inflation rate than lower-weighted components. We use contributions to help us to understand which expenditure categories are making the largest contributions to the rate, accounting for both the weight and price movement. We have calculated contributions to the difference in annual rates. These show the relative contributions of all categories within a given parent category to the difference between the annual rates with and without groceries scanner data. These contributions have been calculated by subtracting the contributions to annual rates for indices without groceries scanner data from the contributions to annual rates for indices with groceries scanner data.

Figure 4 shows that both "food and non-alcoholic beverages" and "alcoholic beverages and tobacco" categories contributed to the differences in the headline annual rate for the Consumer Prices Index (CPI). The contribution of "food and non-alcoholic beverages" was larger, which is in line with the higher weight for this category.

Both "food and non-alcoholic beverages" and "alcoholic beverages and tobacco" categories had higher contributions to annual growth for the series including groceries scanner data between March 2020 and October 2021. Contributions to annual growth tended to be higher with scanner data for "alcoholic beverages and tobacco", and lower with scanner data for "food and non-alcoholic beverages" between January 2022 and December 2022. Contributions to annual growth were generally lower for both categories when groceries scanner data was included between January 2023 and June 2024.

Figure 5 shows the relative contributions of the classes within the food group to the difference in annual rate of the CPI when including and not including scanner data.

Most classes tend to contribute to the difference in a consistent direction. Most categories had a positive contribution to the difference between April 2020 and December 2021, during the coronavirus (COVID-19) pandemic. Most categories had a negative contribution to the difference from January 2022 to June 2024, when inflation was at a historic high for much of the period.

There are some exceptions, including bread and cereals between May 2022 and June 2023, oils and fats in several months of 2022 and 2023, and sugar, jam, syrups, chocolate, and confectionary mostly in 2023. These classes tend to have a positive contribution to the difference in these periods, while other categories have a negative contribution. The positive difference for bread and cereals in this period was caused by breakfast cereals, pizza, and other bakery products like cakes and biscuits. The positive differences for oils and fats in several months of 2022 and 2023 were caused by margarine and similar preparations.

The differences for oils and fats were during a period of particularly acute price increases for vegetables oils. This resulted, in part, from the Russia Ukraine war, which led to disruption in sunflower oil exports and increased demand for other vegetable oils. Our results may indicate that the resulting price increases were better captured by scanner data than our traditional methods.

We undertook more detailed investigations into the causes of differences between growth rates for some chosen consumption segment-level indices.

Figure 6 shows the impact of introducing groceries scanner data on the annual growth rates for the Consumer Prices Index (CPI) "breakfast cereals" consumption segment.

Annual growth was higher for the index including groceries scanner data between June 2022 and June 2023. One period with notably different movements between the two series was January 2023. Annual growth increased between December 2022 and January 2023 for the index including groceries scanner data, but decreased for the series without groceries scanner data over this period. This period was used as a case study to investigate some possible causes of the differences.

Many products contributed to upward index movements in the scanner data that were not represented by any price quotes from the same retailers in the traditional collection. This suggested that the narrower coverage of the local collection was likely one factor contributing to the differences in observed annual growth.

We have several quotes from retailers for which we also have grocery scanner data (which we are using to replace traditional collection quotes in the scenario with groceries scanner data). The price movements for the same products in the scanner data had less negative price movements than in the local collection data.

It could be seen that the traditional price collection took place at a point in the month where prices were lowest for these products because of ongoing promotions. With the scanner data, an average price was calculated over a period that included both the lowest promotional prices and some higher prices. This demonstrates how the calculation of average prices in the scanner data over a longer time can result in prices that are more representative of the whole month.

A second CPI consumption segment investigated was "cheese, cheddar". Figure 7 shows the impact of introducing groceries scanner data on the annual growth rates for this consumption segment.

Annual growth was substantially lower for the index including groceries scanner data across two extended periods. One period with notably different movements between the two series was February 2023. Annual growth decreased for the index including groceries scanner data between January 2023 and February 2023 and increased for the series without groceries scanner data over this period. This period was used as a case study to investigate some possible causes of the differences.

We found that some of the largest causes of negative growth within the groceries scanner data were branded products with large price changes that indicate promotional activity. These branded products had very limited coverage in the traditional price collection, so these negative price movements had limited impact.

This highlights the benefit of the improved coverage of groceries scanner data, which ensures that a much wider range of products and their price movements can be captured.

We aim to publish final impacts of this transformation at the end of 2025. Following our publication, we will decide whether to move these new data and methods into use in live production. If we are satisfied that our data, methods, and systems are ready for live monthly production of these indices, they will first be introduced in the figures for February 2026, published in March 2026. The existing published series will not be revised.

Our broader plans to transform UK consumer price statistics by including new improved data sources and developing our methods and systems are discussed in our Transformation of consumer price statistics: August 2024 article.