Impact of introducing a new data source for shop-type weights on consumer price indices

This article describes the effect of updating the underlying stratum weights in the Consumer Prices Index including owner occupiers' housing costs (CPIH – the Office for National Statistics' (ONS) lead measure of inflation), the Consumer Prices Index (CPI) and the Retail Prices Index (RPI).

From March 2020, we will publish CPIH, CPI and RPI using the Annual Business Survey (ABS) as the new data source for the calculation of shop-type stratum weights. This will replace the shop-type weights derived from our current data source, the Annual Retail Inquiry, which was discontinued in 1999. The weights will also be updated in the next publication of the experimental Household Cost Indices. For a description of our different measures and approaches to inflation, please see Measuring changing prices and costs for consumers and households: March 2018.

This article presents analysis of the effect the new data source would have had, had it been introduced in the period January 2011 to August 2019. Based on the historical analysis, the greatest expected impact of the change on the headline CPIH is a change in the 12-month growth rate of 0.2 in June 2012, and an impact of 0.1 percentage points in 37 out of 103 months, as shown in Figure 1.

Consumer price indices estimate the changes to the total cost of a "shopping basket" of goods and services bought by a typical UK household. Spending on different areas of the basket can vary greatly, therefore the components of the index are "weighted" to reflect their importance, and how much is spent on them in different regions and in different types of shops.

Shop-type weights are used for the stratification of locally collected items. There are four stratum types:

• region

• shop type (multiple or independent)

• region and shop type

• no stratification

Multiples are defined as retailers with 10 or more outlets, and independents are defined as retailers with fewer than 10 outlets.

Shop-type weights were updated annually using the Annual Retail Inquiry (ARI) until its termination in 1999, after which the weights received minor updates using data from the Living Cost and Food Survey (LCF) until 2006. The weights have not been updated since then. This led to a recommendation in an independent review by Paul Johnson (2015) that stratification by shop type should be reviewed with the aim of finding a data source which would allow annual updates.

In September 2019, a paper was taken to the Technical Advisory Panel on Consumer Prices (APCP-T) (PDF, 199.65KB) discussing two possible data sources: the Annual Business Survey (ABS) and the Annual Survey of Goods and Services (ASGS). The ABS is the main business survey conducted by the Office for National Statistics (ONS). It publishes financial information from businesses representing the UK non-financial business economy (roughly two-thirds of the UK economy) and is our preferred data source to use for several reasons.

As the ABS is conducted by the ONS, the data can be obtained easily with no additional cost, and a continuing supply of data is assured. The categories² used in the ABS are slightly less specific than those that were used in the ARI but can be mapped well. The mapping between the ARI and ABS categories can be found in Annex A. They also map well to the Classification of Individual Consumption according to Purpose³ (COICOP) classification system used for CPIH and CPI. As such, there is the opportunity to adapt the stratification in the future to fit more closely with the existing COICOP structure. The ASGS is a relatively new survey and could be used in future work to improve stratification, as described in Section 4, once it is more established.

The ABS does not include a question on outlet counts, which is needed to assess a retailer's shop type, but this was resolved by matching the ABS data to data from the Inter-Departmental Business Register (IDBR), which does include outlet counts. The match rate is high, typically above 99%.

Once the new ABS categories had been mapped to their predecessors, new shop-type weights were calculated for 2011 to 2019. These were processed in the same way as before, except the weights now changed each year. There were some limitations to this, for example, as the ABS categories are not as specific as those in the ARI, there were issues assigning a shop-type weight to specific items (for example, pushchairs). In these instances, items were assigned their old ARI weight for the purpose of this analysis.

Further to this, the ABS data are not published in time to be used in the year to which they relate and are therefore incorporated with a three-year lag. Expenditure data used at COICOP level typically has a two-year lag, but some data are lagged by three years at item level, albeit constrained to more recent data from the LCF survey. Consumer price indices from February 2020 will use the new data source. For March 2020 to February 2021, shop-type weights will be calculated using the 2017 ABS data.

In most cases, a category's multiple-independent split varies by less than 10 percentage points over the nine years studied.

Notes for Background and explanation of changes:

1. UK Consumer Price Statistics: A Review was a wide-ranging review of UK consumer price statistics, commissioned by the UK Statistics Authority following the consultation on the Retail Prices Index (RPI) in 2012.

2. The ABS uses the Standard Industrial Classification: SIC 2007 to classify businesses by type of economic activity.

3. COICOP (Classification of Individual Consumption According to Purpose) is a classification scheme defined by the United Nations (PDF, 2.45MB) used by National Statistical Institutes to group items in their consumer price indices.

To assess the expected impact of this change, we recalculated the Consumer Prices Index including owner-occupier's housing costs (CPIH), Consumer Prices Index (CPI) and Retail Prices Index (RPI) from 2011 to 2019 to understand what the impact would have been, had the change been implemented historically. We used the new shop-type weights for each year calculated using the ABS with a three-year lag. The data were processed using an offline system which resulted in some reconciliation differences. Consequently, a proxy series has been used as comparison with the revised series using the new weights instead of the published CPIH, CPI and RPI.

Figure 2 shows the published CPIH series and compares it with the proxy series that was created for use in the analysis, and Figure 3 compares the CPIH annual growth rates for the proxy and published series, which differ by up to 0.2 percentage points.

Analysis of impact on CPIH

Figure 1 and Figures 4 to 6 present the index values, growth rates and differences for the proxy and revised CPIH series, using the new shop-type weights calculated using the ABS and a breakdown of the classes contributing most to the index differences. There is very little difference between the two indices, though there are periods with some minor differences.

There is not a cumulative difference between the proxy CPIH index series and the revised index series at one decimal place.

Figure 1 shows that the difference between the two series' 12-month growth rates is greatest in June 2012, and then becomes smaller for the remainder of the time period examined. The average of the absolute differences in the annual growth rates for CPIH is very small with a maximum difference of 0.2 percentage points. Figure 5 shows that the largest impact on the one-month growth rates was 0.1 percentage points.

Figure 6 shows the difference between the percentage point contributions for the proxy and revised CPIH. The class which has the largest impact is Garments. This is the only class which has an absolute average impact on the 12-month growth rate of CPIH of more than 0.01 percentage points, while 43 classes have an impact of less than 0.0001 percentage points. This is to be expected as there are not items with shop-type weights in every class, and some classes may only include a couple of items with shop-type weights.

The larger effect that the Garments class has on the 12-month growth rates for CPIH could be because all items in the Garments class have a shop-type weight, whereas Games, toys and hobbies contains some items that don't have a shop-type weight, such as video games.

The impact on CPI and RPI is largely the same as for CPIH and is provided in Annexes B and C.

While the historical impact on the headline CPIH, CPI and RPI is minimal, updating the data source to calculate shop-type weights protects the series against future shocks to expenditure which could have a greater impact. This change also allows us to future-proof against possible changes to the classification structure in the future.

There are further improvements that could be made in the future to better reflect where consumers make their purchases in our consumer price indices.

To complete this work, we matched the Annual Business Survey (ABS) data to the Inter-departmental Business Register (IDBR) to provide an outlet count for the retailers in the ABS. While the match rate was very high, the Technical Advisory Panel for Consumer Prices (APCP-T) thought it could be improved further by making use of the IDBR extract that the ABS draws its sample from. We are exploring this with the aim of improving the match rate.

For this analysis, we have inserted the new data into the existing structure, but for future work we will look to see if we can improve the classification and align it with the Classification of Individual Consumption according to Purpose (COICOP). This could facilitate the use of the new weights for the introduction of alternative data sources.

Another data source which was considered for shop-type weights and brought to the APCP-T was the Annual Survey of Goods and Services (ASGS). One issue with this data source was that it did not ask if the respondent is a multiple or independent retailer. However, as the survey is relatively new, the APCP-T suggested that we ask the ASGS to work towards asking businesses to include this, along with a breakdown of their turnover generated in physical stores and online. This could be used to improve stratification in the future, particularly in relation to the implementation of alternative data sources. If the ASGS can be adapted then it could replace or complement the ABS as our data source in the future.

Figures 7 to 9 present the index series, growth rates and differences for the proxy and revised CPI series using the new shop-type weights calculated using the ABS.

The impact on CPI of updating the shop-type weights annually is also very small. The average absolute impact on the CPI series is 0.03 index points but there is no cumulative impact on the index at one decimal place.

The largest impact on the CPI 12-month growth rates was a change of 0.2 percentage points in 7 out of 103 months.

Figure 9 shows that variation in the impact on one-month growth rates was between 0.1 and negative 0.1 percentage points.

For CPI, the class which contributed most to index differences was Garments.

The impact on RPI is similarly small, as can be seen in Figures 10 to 12. The average absolute impact of the change on the RPI series is 0.1 index points and the overall cumulative impact is 0.1 index points.

The impact on the 12-month growth rates is 0.1 percentage point in 28 months of the series. In most instances, the impact of the new shop-type weights on the 12-month growth rate is an increase of between 0.0 and 0.1 percentage points, as can be seen in Figure 11.

Figure 12 shows that the largest differences between the one-month growth rates for the revised and proxy series were 0.1 percentage points in 19 months.

Coralie Roussel