Producer prices development plan: October 2023

The Office for National Statistics (ONS) produces a range of statistics for both consumer and producer price inflation. The ONS's producer price inflation statistics include both output and input inflation estimates. The output producer price inflation measure comprises domestic and export producer price indices (PPI and EPI, respectively). Our input PPI inflation estimate comprises imported producer input prices (IPI) and domestic producer price indices.

To support this work, the ONS has a programme of research and development aimed at improving and maintaining our range of producer price inflation statistics. The programme will ensure that the statistics continue to meet user needs, make use of new data sources, and follow international best practice.

The UK statistics strategy, Statistics for the public good (2020 to 2025), sets the collective mission for official statistics as: "High quality data and analysis to inform the UK, improve lives and build the future". This is based on four principles which aim to provide clear and insightful economic, social and environmental statistics: “Ambitious, Radical, Inclusive and Sustainable”.

Our development plan reflects how our producer prices development work contributes to the strategy. We contribute by efficiently producing high quality and relevant price statistics to meet user needs, while keeping pace with evolving methods, sources, and digital processes.

The scope of the development plan builds upon the processing and quality assurance improvements implemented following a review by the Office for National Statistics’ (ONS’s) Quality and Improvement Division in 2022. This was undertaken as a result of the pause in publishing producer price indices (PPI) in late 2022 as a consequence of errors identified in PPI weights. For further details, see our Produce Price Index correction notice.

Additionally, the Office for Statistics Regulation (OSR) Producer Price Indices (PPI) Spotlight on Quality report was published in July 2023. The report notes the excellent progress made in bringing PPI in line with methodological best practice over recent years, as well as the improvements made since the series of errors in 2022, and it also highlights areas for improvement.

We welcome the support from OSR and the ONS’s Quality and Improvement Division and agree with the priorities they highlight. The current scope of work is therefore focused on fulfilling the requirements of the recent OSR report. For more information on how this development will address the eight OSR recommendations, see Section 4: Producer price statistics work programme.

The work programme for producer price inflation statistics has been drawn up to reflect the Office for National Statistics’ (ONS’s) strategy and priorities, as well as the requirements laid out by the recent Office for Statistics Regulation (OSR) report.

Some aspects of the work programme also reflect the priority placed on the use of alternative data sources, as found in GOV.UK’s Independent review of UK economic statistics: final report (known as the Bean Review), led by Professor Sir Charles Bean of the London School of Economics.

Short-term priority work

The development programme over the next six months will benefit users as well as fulfil requirements laid out in the Office for Statistics Regulation (OSR) report on producer price indices (PPI). This work will focus on reviewing the volume of data published and how the data are described.

Refining published data

The quality status of PPIs depends primarily on two factors: sample size and overall product coverage. We do not publish any indices where the sample size is so small that our data could potentially reveal the pricing behaviour of specific businesses. Indices such as these are supressed. All indices that we publish will therefore have excellent or good product coverage.

The quality status of the published indices was historically updated on an annual basis coinciding with the renewal of the survey sample. As a result of the pause in implementing updates to the sample, the quality status of our producer price indices has not been updated since 2020. We aim to review and update these by November 2023, which may result in pausing the publication of some indices.

We have also been conducting work on refining index descriptions that accompany all of the producer price indices. This was in response to user feedback aiming to provide a clearer understanding of the purpose of each index and where it fits within the index hierarchy. These were rolled out within our September 2023 publication.

Over the next six months we will be running a consultation process with data users to better understand their needs and rationalise the number of indices that we currently publish. The aim of this is to ensure that the data we publish best meet user needs, are easy to identify and understand, as well as providing a clear narrative in terms of producer price inflation. Any changes to our published datasets will be implemented by May 2024 and will align with data accessibility best practices.

Near-term priority work

The development programme over the next 12 months is based upon work that contributes to the Office for National Statistics' (ONS's) strategy and priorities building upon the short-term priorities which fulfil requirements laid out in the OSR report on PPI. This work will have the primary aim of addressing the core issue of poor sample coverage.

Sample maintenance capability

The sample maintenance programme for the producer price surveys was paused in 2018 allowing resources to be prioritised towards implementing chain-linking methodology. This work was successfully completed in 2020 and planning has now moved to upgrading the sampling system from current legacy systems.

The current sampling methodology is still robust and, therefore, the strategy is to develop a system by summer 2024 that will allow for a rollout of the sample update.

We are also researching methods for aligning the export and import producer price samples with the domestic output PPI sample. The aim of this is to make the process more efficient, simpler to maintain and less burdensome on businesses. It will also ensure closer alignment between our input and output PPI data.

The rollout of any updates to the survey samples is included in our long-term priorities.

Publishing growth rates and weight

To continue to meet user needs we will begin publishing PPI annual and monthly growth rates, as well as the weights of products within the PPI, by May 2024.

Analysis of data revisions

PPI data has a 12-month revision window. This is the period when published data may change because of late returned survey data from businesses. Historically, only the latest three to five months ever saw revisions because of late returned survey data. However, since the first coronavirus (COVID-19) lockdown in March 2020, the survey response from businesses at the point of initial publication has been 10 percentage points lower on average than pre-coronavirus pandemic periods.

This change in business response behaviour has not fully reverted and, therefore, the length of time it takes to gather outstanding data has also extended. This has resulted in more revisions being present within all periods of the revisions window.

We will be analysing revisions in our data so that we can communicate this and become more transparent in how data responses are affecting our published data. We will also look to make recommendations on processing, which will aim to control and minimise the effect of revisions on our published data.

We aim to conclude this work and begin publishing an annual summary of revisions by June 2024.

Non-survey data source review

Most published PPI data are compiled from survey data gathered from manufacturing businesses within the UK. However, for some products, there is non-survey data available (for example, data from the Department for Environment, Food and Rural Affairs, the Marine Management Organisation and the Department for Energy Security and Net Zero). This supplements our survey data, achieving more comprehensive coverage within our published data.

Over the next 12 months, we will be reviewing these data sources and updating our Quality and Methodology Information (QMI) with information on how these data sources are used to calculate the final estimates for PPI.

We will also be exploring additional non-survey data sources that could be used to reinforce PPI data and improve the sample coverage in future. All this work will be concluded by July 2024 and reflected in updates to our QMI.

Long-term priority work

The priorities beyond the next 12 months include the long-term work that contributes to the ONS strategy and priorities, as well as meeting the requirements included within the OSR report. The delivery of a selection of these is dependent on wider ONS development projects, such as legacy system reduction and the move to online data collection.

Sample maintenance roll-out

Once the initial sampling system capability is in place (see the Sample maintenance capability section of this article), we intend to complete a targeted rollout of the updated sample by July 2025. This update to the sample will be implemented in waves initially targeting indices that either require the most attention in terms of sample maintenance, or indices that have the most importance to users.

Details of this rollout will be published in an update to these plans in July 2024.

Sample size review

The last review of sample sizes for PPI was conducted in 2005. We will look to repeat this review for PPI and extend it to also cover the export producer price index (EPI) and imported producer input price index (IPI) samples. This review will be concluded by July 2025 and any intended action will be published at this point.

Data validation

At present the validation process for all PPI survey data is performed at the point of collection and focuses on identifying notable movements in price for individual products. Although this approach does ensure that all data included in the PPI estimates are suitable, this process is not efficient or as effective as it could be. Therefore, we aim to research additional validation methods such as those that focus more on outlier detection or price movements not aligning with price behaviours of similar products. Recommendations resulting from this research (to be completed by July 2024) will ensure that validation methods used for PPI data are sustainable, while also providing users with assurance on the quality of published data.

All data collection and validation processing are currently performed using our existing legacy system. The ONS is currently undertaking a system transformation programme which will migrate our data collection onto a more sustainable system. The implementation of any recommended changes to data validation will be included in the system requirements.

The sequencing and delivery of improvements to PPI as part of the wider legacy system reduction work beyond March 2025 will depend on funding and prioritisation discussions with stakeholders. When we have more defined delivery dates for this programme of work, this plan will be updated to include these details.

Imputation review

Our existing method of imputation is described in our QMI. However, since the coronavirus (COVID-19) pandemic, the overall response rate has fallen notably at the point when we publish our initial producer price inflation estimates (see the Analysis of data revisions section of this article). As a result, on average, 26% of our initial inflation estimate is based upon imputed data; prior to the coronavirus pandemic this would have been approximately 16%. Within indices where there is a limited sample, this can have an effect of amplifying the impact of specific product price movements.

This review will aim to establish whether our current approach still meets the expected methodological requirements when calculating producer price inflation estimates.

The review will be complete by July 2024 and will include any recommendations for change that may emerge from the review process.

The Office for Statistics Regulation (OSR) report contains the following eight requirements and sets out expected delivery dates for each.

Requirement 1: sampling

OSR requirement

To improve the quality of the producer price inflation statistics (PPIs) and ensure that they meet user needs, the Office for National Statistics (ONS) should undertake a review of the necessary sample size and sample optimisation for the producer price indices (PPI), export producer price indices (EPI) and imported producer input price indices (IPI) surveys by July 2024, and update the samples accordingly by July 2025. In the meantime, the ONS should consider what remedial changes it can make to improve sampling arrangements sooner than 2025.

ONS response

For full details, see the Sample maintenance capability and Sample size review sections of this article.

Requirement 2: publish plans

OSR requirement

To safeguard the quality of PPIs, the ONS should publicly commit to clear and achievable transformation plans for developing a robust, flexible, and sustainable producer price inflation statistics system. This should enable Reproducible Analytical Pipeline (RAP) principles to be applied throughout and allow new sources to be used and new methods to be implemented. The ONS should publish and promote the plans as part of the wider PPI improvement plans by September 2023.

ONS response

We have completed this requirement in publishing this article, which sets out our plans for improvements to the PPI systems.

Requirement 3: data validation methods

OSR requirement

Within six months of moving to the Statistical Processing Platform, the ONS should review the PPI data validation processes and checks to ensure they provide an appropriate level of quality assurance and are adaptable to the prevailing general level of price increases.

ONS response

For full details, see the Data validation section of this article.

Requirement 4: revisions review

OSR requirement

To improve its understanding of revisions and minimise their impact on quality, the ONS should carry out a revisions analysis every year. Where revisions are found to be significantly different from zero, the ONS should investigate their source and, where necessary, make appropriate improvements to the methods for producing PPIs.

ONS response

For full details, see the Analysis of data revisions section of this article.

Requirement 5: non-survey data sources

OSR requirement

To ensure the continued suitability of data sources used to produce PPIs, by July 2024 the ONS should review the suitability and quality of all current data sources and improve its understanding of the quality assurance carried out by data suppliers. To help users understand how PPIs are compiled, the ONS should add a high-level process map to quality documentation explaining how the different sources contribute to the final estimates.

ONS response

For full details, see the Non-survey data source review section of this article.

Requirement 6: imputation

OSR requirement

To ensure the best available methods are being used, by July 2024 the ONS should review its imputation methods, including assessing whether they are still fit for purpose and not introducing bias.

ONS response

For full details, see the Imputation review section of this article.

Requirement 7: refine published data

OSR requirement

To maximise the usefulness and quality of the published indices, and optimise the use of available resources, the ONS should rationalise the number of indices produced by July 2024. It should take into account user needs and sample size limitations.

ONS response

For full details, see the Refining published data section of this article.

Requirement 8: update Quality and Methodology Information (QMI)

OSR requirement

To enhance transparency and provide reassurance to users about quality, by July 2024 the ONS should ensure that its published information about data sources, methodology and quality assurance covers all aspects of the production of the statistics and is suitable for a range of users. The ONS should review and update this information whenever needed to reflect current processes.

ONS response

Our QMI will be updated whenever any processing or methodological change is applied because of the work programme detailed in this article. All updates relating to these will be completed by July 2024.