Monthly Business Survey: improving the method for survey non-response

• The imputation methodology for the Monthly Business Survey (MBS), which underpins the current price series in Index of Production (IoP) and Index of Services (IoS) has been improved using the internationally-recognised best practice method; this is now consistent with all other short-term economic output indicators.

• Data revisions as a result of this change go back to January 2016.

• The impact of the changes to IoP have not revised previously published gross domestic product (GDP) quarterly growth rates to one decimal place.

• The impact of the changes to IoS have not revised previously published GDP quarterly growth rates by greater than positive or negative 0.1%.

2.1 Overview

When one or more values are missing in the response returned by a sampled business, by simply discarding this business it may introduce bias or affect the representativeness of the results. Office for National Statistics (ONS) uses a process called imputation to deal with missing responses from businesses. This method preserves all cases by replacing missing data with a probable value based on other available information. Once all missing values have been imputed, the complete dataset can then be analysed.

Ratio imputation is the most common imputation method used in ONS business surveys when the contributor has a valid value in the previous period and is used by both the Retail Sales Index and construction. It applies a period-on-period movement ratio to the contributor’s previous period value to create a value for the missing data item. There are two standard approaches to ratio imputation – mean of ratios, and ratio of means.

This article explains the changes to the imputation methodology introduced to the Monthly Business Survey (MBS). The MBS is used to produce the outputs for the Index of Production (IoP) and Index of Services (IoS) and the imputation method will be updated from the mean of ratios method to the ratio of means.

2.2 Ratio imputation methods

The imputed value y*_i,t for current period t and business i, is given by calculating a growth factor R and multiplying it by y_i,t-1.

The method for calculating R differs between mean of ratios and ratio of means. Both methods are based on businesses that returned data in both current and previous periods. Businesses are grouped into imputation classes (impclass) based on the UK Standard Industrial Classification of Economic Activities 2007. R is then calculated for each of these imputation classes.

Mean of ratios

Can apply two refinements:

• Trimming – due to sensitivity to extreme ratios

• Weighting – to account for growth in previous period

Ratio of means

Therefore:

No trimming parameters or weighting required.

The previous method, the mean of ratios, used trimming to remove the smallest and largest ratios from the growth factor calculation. This would exclude the top and bottom 10% of businesses and thus remove the largest period-on-period changes for a given industry. The new method, the ratio of means, does not use trimming as it is considered a more robust method because it includes all of the data returned. The result of this means that the naturally occurring peaks and troughs, for example, Easter could be slightly larger. The new methodology means that imputed businesses are more representative of the trends in their industry.

2.3 Why is the imputation method changing?

Over the four short-term economic indicators (STEI), Construction in Great Britain and Great Britain retail sales have already transitioned to the ratio of means method. The Index of Production and Index of Services are the final two to change. Analysis over the years has identified that the ratio of means method is expected to produce less non-response bias than the mean of ratios method because it includes all of the data returns and does not involve trimming. The growth factors applied in the mean of ratios method are sensitive to outlying ratios (that is, large and small) and require regular reviews to ensure a sufficient level of trimming is being conducted to achieve a robust growth factor. The growth factor (R) is robust for the ratio of means method and requires no trimming or weighting. The ratio of means method is now recognised in the Recommended Practices for Editing and Imputation in Cross-Sectional Business Surveys EDIMBUS manual (see C.4.2) as international best practice for imputation when the contributor has a valid value in the previous period.

3.1 Overview

In the July 2017 publications of Index of Production (IoP) and Index of Services (IoS), the Monthly Business Survey (MBS) results compilation system was run with the usual mean of ratio methodology to generate total turnover by industry for production and services industries, including export turnover associated with the production industries.

The methodology within the MBS results system was then changed to ratio of means, and results were run again back to January 2016. We compared the turnover and export datasets and did not identify significant changes. We took the decision to run results back to January 2016, as this is the first period after the latest input-output supply use balanced year of 2015 that is consistent with UK National Accounts in Blue Book 2017. For further information on the supply use balancing see section 3: Background to supply and use balanced estimates of annual current price GDP in Commentary on supply and use balanced estimates of annual GDP: 1997 to 2014.

The new imputation methodology has been implemented for the July 2017 publications of IoP and IoS meaning the July indices and growth rates are based on the new method. The change of imputation methodology was applied to MBS data from January 2016. Data revisions as a result of this change were stored up in the IoP and IoS data compilation systems as usual practice in line with the National Accounts Revisions Policy. However, for IoP these were not visible to users in the July 2017 publication as the IoP does not open for revisions in their pre-Blue Book publication. These stored-up revisions will be shown in the IoP August 2017 publication, to be released on 10 October 2017 and they will be consistent with the IoP revisions published in Quarterly national accounts, 29 September 2017 and Blue Book 2017 on 31 October 2017. When the revisions to IoP and IoS are shown in these forthcoming releases, they will contain normal revisions as well as the changes to imputation methodology.

The MBS data containing the new imputation methodology ratio of means were published in Turnover in production and services industries (TOPSI) on 8 September 2017. TOPSI is a publication generated from the MBS turnover and export data, publishing total turnover by industry. The data within this publication underpins the current prices series in the Index of Production and Index of Services.

TOPSI is usually only open to take on the latest 13 months of data, for example, July 2016 to July 2017. However, in TOPSI July 2017, revisions were open from January 2016, that is, 19 months, in order to take on and publish the results of the change to imputation methodology. Revisions prior to July 2016 were solely from the change to imputation methodology. Meanwhile revisions from July 2016 to June 2017 contained both revisions to imputation methodology as well as normal data revisions due to late responses and revised data.

An impact analysis was produced to show how this methodological change affects the outputs of the IoP and IoS and as a result the impact this will have on gross domestic product (GDP). The quarterly presentation of data is for GDP purposes. The monthly profile has been revised using all survey data points to January 2016, which have, for example, slightly increased evident peaks and troughs associated with normal seasonal variations.

3.2 Index of Production

The majority of data used to compile the manufacturing sector, and thus the Index of Production (IoP), is collected via the Monthly Business Survey (MBS). The MBS samples around 6,000 businesses every month for the Index of production. The data collected are turnover excluding Value Added Tax (VAT) and exports for some applicable industries. The change of imputation methodology has been applied to both turnover and export turnover questions.

Revisions have only impacted on two of the four IoP main sectors; these are C manufacturing and E water and waste; this is because the majority of the data in these sectors are collected by the Monthly Business Survey. Sector B mining and quarrying has limited MBS source data (it is collected for industries 07 and 08, which have a low weight) and is made up mainly of volume data supplied by the Department for Business, Energy and Industrial Strategy (BEIS) and sector D energy supply is made up wholly of data supplied from BEIS and contains no MBS data.

Revisions to previously published growth rates caused by imputation methodology for total IoP have been positive or negative 0.1% per quarter (Table 1) apart from in Quarter 1 (January to March) 2017 where we saw a larger difference of negative 0.2%. The revisions to total IoP due to the introduction of the new imputation methodology, ratio of means, have been mostly downward.

The impact of the changes to IoP have not revised previously published GDP quarterly growth rates to one decimal place.

3.3 Index of Services

The data used to compile the Index of Services (IoS) is split between data collected via the Monthly Business Survey (MBS) and other administrative data sources. The MBS samples around 27,000 businesses every month for the Index of Services. The data collected is turnover excluding Value Added Tax (VAT). The change in imputation methodology has been applied to the turnover question.

Revisions have impacted all four of the main IoS sectors. The largest revisions came from distribution, hotels and catering in Quarter 1 (January to March) 2016, which was revised down 0.3% and transport, storage and communication in Quarter 1 2016, revised down 0.4%.

Revisions to previously published growth rates caused by imputation methodology for total IoS have been positive or negative 0.1% per quarter (Table 2). The revisions to total IoS due to the introduction of the new imputation methodology, ratio of means, have been mostly downward.

The impact of the changes to IoS have not revised previously published GDP quarterly growth rates by greater than positive or negative 0.1%.

Mark Stephens, Iolo Tomlinson, Gemma Rabaiotti; Office for National Statistics