Business Impact of Coronavirus (COVID-19) Survey: preliminary weighted results

• We have explored various best practice methods for weighting Business Impact of Coronavirus (COVID-19) Survey (BICS) results, including by turnover, employment and count (that is, to reflect the number of all UK businesses).

• Weighting by turnover or employment has minimal difference when compared to the currently published headline unweighted BICS estimates, such as on trading status, workforce status and financial performance.

• Weighting by count of business has the most significant difference when compared to the currently published unweighted BICS estimates.

• Weighting by count provides a representative assessment for all UK businesses as it scales up responses to reflect all UK businesses, of which most are small businesses with between zero and nine employees.

• More detailed weighted estimates will be available as part of the Wave 13 BICS release, to be published on 24 September 2020.

The sample design for the Business Impact of Coronavirus (COVID-19) Survey (BICS) was reviewed and refreshed for Wave 7 and went live on Monday 15 June, and it has been the basis for all waves since. (Section 5: Sample size of BICS, Waves 6 to 12 provides an overview of the current sample design by size band.)

Fortnightly questionnaires go out to approximately 24,500 businesses, and the sample redesign at Wave 7 significantly improved our coverage of the smaller-sized businesses. This improvement to our coverage has provided a suitable foundation for weighting to be applied to the currently unweighted estimates.

Currently, unweighted BICS results mean that we can only make inferences about UK businesses in our sample that have responded. Weighting BICS responses will enable us to make inferences about all UK businesses, not just those in the sample and that have responded. This inference requires weighting and imputation methods to account for both unsampled and non-responding businesses. This section summarizes the weighting methods we have explored, with a more detailed methodological overviews available in Section 4: Methodology in detail.

Different weighting methods are appropriate depending on the type of analysis and policy questions being considered, and Section 6: Selected weight for each BICS table highlights which weights we aim to publish for each BICS table.

Weighting by count

Weighting by count scales up responses in BICS to be representative of all businesses in the UK. It scales up responses for all businesses that have between 0 and 249 employees, to the point where the counts of all businesses of this size in the UK are represented. The size band of greater than 250 employees is completely enumerated, so no weighting is applied.

The weight applied to each response in qualitative questions (for example, on trading status) is based on a standard expansion estimation; for example, where the weight applied is the ratio between the UK population size of a stratum (that is, a group of businesses with the same characteristics based on Standard Industrial Classification (SIC) 2007 industry, employment size and country) and the responding sample size in a stratum.

Of the weighting methods we have considered, weighting by count results in the largest differences to the currently published headline BICS results (for example, proportion of businesses that are currently trading); this is mainly because of the emphasis of the experiences of small businesses. Responses from businesses with zero to nine employees are scaled up the most, given that they make up 90% of the total number of businesses in the UK but only around 10% of respondents in the BICS (see Table 1 in Section 3: Impact of different forms of weighting).

Weighting by count provides a good overview of the impact of businesses regardless of their size, allowing for the experience of small businesses to be better represented in headline results (this is particularly important for responses such as “currently paused trading” where small businesses tend to dominate).

To see the difference small businesses make to headline results, see Section 7: Impact of weighting on businesses with 10 or more employees. It shows how when removing the smallest of businesses (that is, businesses with zero to nine employees), the difference between unweighted and weighted estimates significantly decreases.

Weighting by turnover

The value of turnover is derived using the percentage questions asked on the BICS and registered turnover in the Inter-Departmental Business Register (IDBR), which is used to construct the BICS sampling frame. In effect, businesses with larger turnover are given greater emphasis in results. Once the value of the turnover of responding businesses is derived, a standard ratio estimation is then used to calculate the weight applied to a particular stratum. By using turnover as the auxiliary variable, ratio estimation corrects for any imbalances in the selected sample that arise through random chance or non-response.

While turnover is often associated as a proxy for economic output, it is not in this context comparable to gross domestic product (GDP). See Section 4: Methodology in detail for a more detailed methodological overview on weighting BICS by turnover.

Of the weighting methods we have considered, weighting by turnover results in very small differences to currently published headline BICS results. This is mainly because of medium to large businesses already having a higher proportion of their businesses represented in unweighted BICS responses than smaller businesses (see Table 1 in Section 3: Impact of different forms of weighting).

Weighting by employment

We first derive counts of employment from the percentage questions asked on the survey (such as on percentage of staff furloughed) and multiply these percentages by the registered employment figure recorded in the IDBR at the time of the sample selection. In effect, the percentages reported by businesses with larger employment sizes are given greater emphasis in the results. These counts are then weighted using standard ratio estimation applied to a particular stratum. By using employment as the auxiliary variable, ratio estimation corrects for any imbalances in the selected sample that arise through random chance or non-response.

See Section 4: Methodology in detail for a more detailed methodological overview on weighting BICS by employment.

Of the weighting methods we have considered, weighting by employment (like turnover) results in very small differences compared with the currently published headline BICS results. This is mainly because of medium to large business already having a higher proportion of their businesses represented in unweighted BICS responses than smaller businesses (see Table 1 in Section 3: Impact of different forms of weighting).

Imputation

While results published in this article do not include imputation, work is ongoing to develop imputations for non-responding businesses with more than 250 employees. Based on our methodology, imputations are expected to have a small impact on weighted by count results as businesses with more than 250 employees make up a small proportion of all UK businesses. Imputation is only applied to the larger businesses (those with more than 250 employees) as all these businesses in this population are sampled, and the use of imputation allows for a more representative estimate of these businesses based on their response characteristics from earlier waves where available.

There are various forms of imputation dependent on if the business responded in a previous wave or not, and these are outlined in detail in Section 4: Methodology in detail.  

Trading status

Figure 1 shows the trading status of businesses over time, comparing unweighted results with estimates using all three forms of weighting: count, turnover and employment.

Currently published unweighted estimates show that the proportion of businesses that were currently trading gradually rose from 86% in Wave 7 (1 to 14 June 2020) to 96% by Wave 12 (10 to 23 August 2020). Weighting by either turnover or employment results in very similar proportions to unweighted estimates throughout this time period. This is to be expected, given that unweighted estimates consist of responses mainly from medium to large businesses and therefore results mainly reflect the experiences of medium to larger businesses. Weighting by turnover or employment also has a similar effect as it in effect gives a greater emphasis to the experiences of larger businesses (that is, the larger their turnover or employment level, the larger their weight).

Weighting by count of businesses, however, has a notable downward impact compared with the unweighted proportion of businesses that are currently trading over time. While the overall trend over time is largely similar, using weighted estimates by count, the proportion of businesses that were currently trading gradually rose from 66% in Wave 7 to 83% by Wave 12, compared with unweighted results of 86% and 96%, respectively.

The reason for this is the nature of weighting by count. This approach scales up responses in each of the fortnightly surveys to be representative of all UK businesses, regardless of size or market presence. This is further explained in Figure 1.

Column A in Table 1 shows how the smallest businesses (that is, businesses with zero to nine employees) make up 89.7% of the total UK business population but only around 10% of responses since Wave 7. In effect, weighting by count scales up the responses in the smallest size band the most and, as a result, scales up the experience of small businesses throughout every response category in a question over that of other businesses (Figures 1 and 2 are examples of this).

Table 2 shows how businesses that were currently trading were more often larger businesses (those with more than 250 employees) while those that had temporarily closed or paused trading were mainly small to medium businesses (those with fewer than 250 employees). As a result of the experiences of small to medium businesses being scaled up the most when weighting by count, the number of businesses that had temporarily closed or paused trading is increased to a greater degree than the number of businesses that were currently trading; this results in a higher proportion of businesses that had temporarily closed or paused trading and a reduction in the proportion of businesses that were currently trading.

Overall, weighting by turnover or employment allows a more appropriate assessment on the impact of the coronavirus (COVID-19) pandemic on the wider UK economy, while weighting by count allows for a more representative assessment of the impact of the pandemic on UK businesses.

Financial performance

Figure 2 shows the proportion of businesses that have been experiencing an increase, decrease or no change in their business turnover (that is, financial performance) since the beginning of June through to August 2020. It compares unweighted results with estimates using all three forms of weighting: count, turnover and employment.

Currently published unweighted estimates show that the proportion of businesses reporting a decrease in turnover in the previous two weeks gradually fell from 64% in Wave 7 (1 to 14 June 2020) to 47% by Wave 12 (10 to 23 August 2020). Weighting by either turnover or employment results in very similar proportions to unweighted estimates throughout this time period and across the different response categories.

However, estimates weighted by count produce slightly different results because this weighting method scales up the experiences of small businesses the most, and it is likely that the experiences of small businesses are different to that of larger businesses. The proportion of businesses with turnover unaffected is at a higher level throughout the waves, and there is a reduction in the other response categories as small businesses dominate this response category the most.

Workforce

Figure 3 shows the breakdown of businesses’ workforce status over time, since the beginning of June through to August 2020. It compares currently published results on proportions furloughed and working normally or remotely with estimates using all three forms of weighting: count, turnover and employment (grossed to provide a UK-wide estimation).

Currently published workforce proportions are based on the responses provided by businesses, which are then apportioned to derive proportions of employees in those businesses using the employment recorded for each reporting unit on the Inter-Departmental Business Register (IDBR). The currently published apportionment of workforce methodology used for these data does not involve weighting to make a UK-wide estimation; therefore, findings have not been fully representative of the UK workforce. Using this currently published method, proportions of the workforce furloughed within responding businesses that had not permanently stopped trading fell from 23% in Wave 7 to 11% in Wave 12, while the proportion working at their normal place of work or working remotely rose from 72% in Wave 7 to 85% in Wave 12.

Using the weighting by employment estimate allows for the grossing up to provide a UK-wide estimation. Using this method, results are broadly similar to those currently published apportioned estimates with the proportion of the workforce furloughed falling from 32% in Wave 7 to 16% in Wave 12, while the proportion working at their normal place of work or remotely rose from 66% in Wave 7 to 82% in Wave 12. The main reason for the small differences between the apportioned and weighted estimates is that apportioned estimates do not take into account non-responders, while weighted estimates do.

Weighting by either turnover or count results in almost identical proportions compared with employment weighted estimates throughout this time period and across the different response categories.

Weighting by count

The Business Impact of Coronavirus (COVID-19) Survey (BICS) sample design selects a random sample from the four smallest size bands, that is, businesses with 0 to 9, 10 to 49, 50 to 99 and 100 to 249 employees, across all industries (note that businesses with more than 250 employees are separately completely enumerated within the BICS sample). With a responding sample size of “n” in a stratum (that is, a group of businesses in the same size band and industry where the industry grouping is based on Standard Industrial Classification (SIC) 2007) and a population size of “N”, the methods required are as follows.

For weighting by count, the weight applied to each response is a standard expansion estimation:

where w_h is the weight in stratum h, N_h is the population size in stratum h, and n_h is the responding sample size in stratum h.

Weighting by turnover

For weighting by turnover, the weight applied to each response from the four smallest size bands is a standard ratio estimation:

where the summation over N means for all businesses in the stratum, and over n means for all businesses in the responding sample.

In other words, it is the ratio of the population’s registered turnover over the responding businesses’ registered turnover, calculated and applied per stratum.

Weighting by employment

For weighting by employment, the weight applied to each response from the four smallest size bands is a standard ratio estimation:

where the summation over N means for all businesses in the stratum, and over n means for all businesses in the responding sample.

In other words, it is the ratio of the population’s registered employment over the responding businesses’ registered employment, calculated and applied per stratum.

Imputation

The BICS sample design also selects all businesses with employment greater than 250, across all industries (based on SIC 2007). In this case, the non-responding businesses in each wave need to be imputed. While the imputation design is at five-digit SIC level, imputation “classes” may be at two-digit level or industry section if not enough responses exist (that is, if there are fewer than 10 responses).

For qualitative questions, the imputation method will be “hot deck” imputation, where a random selection is made from what is called a “hot deck” of valid responses already available. The details of the method depend on whether there is a previous response for the non-responder.

Where there is no previous response or there is a new question, the “hot deck” is all responses in the imputation class, in the current period. For example, if response options are A, B or C, and the numbers responding to each were A:5, B:10 C:35, the random selection has probability of 0.1 of selecting A, 0.2 of selecting B and 0.7 of selecting C.

Where there is a previous response (that is, in the wave prior), the “hot deck” is all responses in the imputation class, in the current period, for businesses with the same¹ response in previous period as the non-responder. For example, if non-responder chose “A” in the previous period, then the “hot deck” is all businesses that chose “A” in the same previous period and that also responded in the current period.

For quantitative questions, the imputation method will be ratio of means, which calculates an imputation link and applies it to the non-responder. The details of the method depend on whether there is a previous response for the non-responders. Where there is no previous response or there is a new question, the link is:

where y is the response for a responding business, and summation is over all responses in the imputation class (i).

For example, if the imputation link is 0.2, this is multiplied by the Inter-Departmental Business Register (IDBR) employment of the non-responder to impute the response.

Where there is a previous response (that is, in the wave prior) the link is:

where t refers current period, t - 1 to the previous period, and summation is over all matched responses, that is, only including businesses that responded in both periods.

For example, if the imputation link is 1.2, this is multiplied by the previous response of the non-responder to impute the response.

Notes for: Methodology in detail

1. For workforce percentage questions, the hot deck will have to be defined based “On furlough leave”.

The sample design for the Business Impact of Coronavirus (COVID-19) Survey (BICS) was reviewed and refreshed for Wave 7 and went live on Monday 15 June, and it has been the basis for all waves since. Table 3 provides an overview of the current sample design by size band.

Different weighting methods are appropriate depending on the type of analysis and policy questions being considered, and Table 4 highlights which weights we aim to publish for each Business Impact of Coronavirus (COVID-19) Survey (BICS) table.

Tables 5 to 7 show results of weighting when the smallest businesses (that is, businesses with zero to nine employees) are removed from headline Business Impact of Coronavirus (COVID-19) Survey (BICS) results, compared with currently published unweighted and weighted results for all size-bands. These highlight that the main driver of the level difference between unweighted estimates and estimates weighted by count are small businesses.

Notes

1. Final results Wave 7 to 12 of the Office for National Statistics (ONS) Business Impact of Coronavirus (COVID-19) Survey (BICS).
2. Only results from businesses that reported they were currently trading are presented.
3. For presentational purposes, trading for more than the last two weeks and started trading again within the last two weeks after a pause in trading have been combined to currently trading.
4. Businesses were asked for their current trading status and so responses will be from the point of completion of the questionnaire.
5. All businesses includes businesses across all industries and employment size bands (that is, with 0 to 9, 10 to 49, 50 to 99, 100 to 249 and 250 or more employees).

Notes

1. Final results, Wave 7 to 12 of the Office for National Statistics (ONS) Business Impact of Coronavirus (COVID-19) Survey (BICS).
2. Only results from businesses that reported they were currently trading are presented.
3. For presentational purposes, increased turnover categories and decreased turnover categories have been combined, while “not sure” has been omitted from the figure but is available in the accompanying dataset.
4. All businesses includes businesses across all industries and employment size bands (that is, with 0 to 9, 10 to 49, 50 to 99, 100 to 249 and 250 or more employees).

Notes

1. Final results Wave 7 to 12 of the Office for National Statistics (ONS) Business Impact of Coronavirus (COVID-19) Survey (BICS).
2. Only results from businesses that have not permanently stopped trading are presented.
3. The currently published figures represent the proportion of responses to each question from businesses, apportioned using the employment recorded for each Reporting Unit on the Inter-Departmental Business Register (IDBR).
4. “Other” is a combination of the other options businesses had to provide a proportion for: “off sick or in self-isolation due to coronavirus (COVID-19) with statutory or company pay”, “made permanently redundant” and “other”.
5. All businesses includes businesses across all industries and employment size bands (that is, with 0 to 9, 10 to 49, 50 to 99, 100 to 249 and 250 or more employees).

This article covers the general impact that the different weighting methods have on some of our main currently published unweighted Business Impact of Coronavirus (COVID-19) Survey (BICS) variables, ahead of releasing a full suite of weighted results on a regular basis. Furthermore, this preliminary analysis looks at the impacts of weighting without taking into account imputation for non-responding businesses with more than 250 employees.

Our methodology will allow estimates to be derived for the different size bands. For example, weighted estimates for small (0 to 9 employees), medium (9 to 249 employees) and large (more than 250 employees) businesses. This can also allow the creation of specific combinations, for example, weighted estimates for more than 10 employees for specific variables (see Section 7: Impact of weighting on businesses with 10 or more employees, where preliminary results are available for the main questions).

While regional weighted estimates can be derived using the same methodological approach, we are continuing further work in this area. Weighted regional estimates may separately be available via the devolved administrations on a case by case basis.

While different weighting methods have their value and use, they are not all necessary for every table in the fortnightly BICS tables. Different policy questions can be aided using different weighting methods on the same variable. We plan to weight by count for most tables in the BICS, although we will publish some variables based on weights taking account of turnover or employment, whichever is most appropriate to the question.

Table 4 (see Section 6: Selected weight for each BICS table) shows our initial approach to which weighting method will be used for each table in the BICS. We will present a selection of these weighted estimates for the next fortnightly BICS release to be published on 24 September 2020.

Acknowledgements

Freddy Farias Arias and Emily Hopson, Office for National Statistics.

The authors would like to acknowledge the contributions of Gary Brown, Lewis Edwards, Jon Gough and Craig McLaren