An overview of and comparison between Annual Survey of Hours and Earnings (ASHE) and Average Weekly Earnings (AWE): 2017

The Office for National Statistics (ONS) has two main sources of earnings statistics: the Annual Survey of Hours and Earnings (ASHE) and Average Weekly Earnings (AWE), which is produced from the Monthly Wages and Salaries Survey (MWSS). In summary, this article will:

• present an overview of ASHE and AWE through addressing their methods, sampling, survey strengths and limitations
• highlight which source is better suited for certain types of analysis and how average pay growth for both sources can be affected by compositional changes in the workforce
• analyse the data movements of both whole economy time series between 2005 and 2016

2.1 Annual Survey of Hours and Earnings (ASHE)

The Annual Survey of Hours and Earnings (ASHE) is our main measure of earnings in terms of structural statistics; it provides information about the levels, distribution and make-up of earnings and hours paid for employees in the majority of industries and occupations across the UK. It is also the lead source of estimates for the gender pay gap and the number of jobs paid below the National Minimum Wage or National Living wage. The level of detail available, reflecting a large sample size and more data collected, means it is published on an annual basis.

Estimates are presented by a number of breakdowns, including sex, age, occupation, industry and region. Statistics are published for both the median and the mean, the headline statistics for ASHE being based on the median. The median is the preferred measure of average earnings as it is less affected by a relatively small number of very high earners and the skewed distribution of earnings. It therefore gives a better indication of “typical” pay than the mean.

Published headline ASHE estimates for weekly earnings relate to gross earnings for full-time¹ employees, on adult rates and whose earnings for the survey pay period were not affected by absence. They do not include those whose earnings were reduced because of, for example, sickness. The estimates relate to a reference point in April each year.

The ASHE headline measure is not solely a measure of rates of pay and can be affected by changes in the composition of the workforce. For instance, all other things being equal, an increase in the relative number of employees in highly paid industries will cause average earnings in ASHE to rise. The following section explains the factors affecting the change in average earnings in more detail.

2.2 Average Weekly Earnings (AWE)

Average Weekly Earnings (AWE) is our lead indicator of short-term changes in earnings. It is designed to capture monthly changes in the average weekly earnings of employees in Great Britain. AWE is based on the Monthly Wages and Salaries Survey (MWSS), which covers employees working in businesses with 20 or more employees in all industrial sectors in Great Britain (an adjustment is made for smaller businesses using ASHE data). As it is collected monthly, there is much less detail than the yearly ASHE.

AWE, for any given month, is the ratio of estimated total paid in wages and salaries for the whole economy, divided by the total number of employees. Therefore, AWE is the mean rate, as opposed to the headline median estimate reported in ASHE. As with ASHE, AWE is not a measure of rates of pay and can be affected by changes in the composition of the workforce.

Estimates are available for total pay (which includes bonus payments) and for regular pay (which excludes bonus payments) at sector and industry level. Estimates are available in both nominal terms (not adjusted for inflation) and real terms (adjusted for inflation).

Unlike the ASHE headline measure, AWE does include the earnings of those employees who work part time and employees whose earnings were reduced for any reason.

Further information for both ASHE and AWE are provided in Annex A and additionally in the Guide to Sources of Data on Earnings and Income. For further information on ASHE specifically, see ASHE Guidance and the ASHE Quality and Methodology Information and for AWE specific information, see the AWE Quality and Methodology Information.

Notes for: Overview of the sources

1. Those employees working over 30 hours in the reference week.

A common misconception of the headline Annual Survey of Hours and Earnings (ASHE) and Average Weekly Earnings (AWE) figures is that they are measures of pay rises. However, they are both designed to estimate earnings of all employees in the economy at a single point in time and therefore measure the change in average pay, rather than the average of changes in pay. This is the reason why average pay can increase without anyone having had a pay rise.

As noted in the overview of the two sources, both ASHE and AWE earnings estimates can be affected by changes in the composition of the workforce, for example:

• increases or decreases in the number of part-time or full-time employees
• changes to the number of hours employees work
• employees entering or leaving the workforce¹

These changes in the workforce mean that the averages may not be measuring changes within each series on a “like for like” basis. A worked example demonstrating this is included in Annex B.

It is important to ensure users are using the most appropriate earnings estimate for their needs as each will be better suited to some uses over others. For example, ASHE is the preferred series for measuring levels of pay due to its granularity, whereas AWE is more suitable for measuring rates of change as it is a more frequent series. We therefore provide, at Annex C, a short guide on the best measure to use depending on the issue being investigated.

Notes for: What do ASHE and AWE measure?

1. The infographic in Section 4 of the “Understanding average earnings for the continuously employed” article gives an illustration of the impact that employees entering and exiting the workforce has on median earnings.

4.1 ASHE

4.1.1 Strengths

One of the main uses of Annual Survey of Hours and Earnings (ASHE) is to analyse the distribution of earnings in the UK. While this is also possible using the Labour Force Survey, ASHE is considered to be a more accurate source of information on earnings as the information is provided by employers rather than being self-reported by employees.

ASHE also benefits from a large sample size (1% of the employee workforce on PAYE schemes) which, coupled with the array of individual and geographic characteristics that are also gathered, allows for more detailed analysis than other sources of earnings data.

ASHE also benefits from sampling the same employees over time which gives it a longitudinal aspect which isn’t present to the same extent on other surveys of earnings.

4.1.2 Limitations

The most obvious limitation of ASHE is that, due to it being an annual survey, it is not as timely as other measures of earnings. Whereas Average Weekly Earnings (AWE) has a six to seven-week lag, the lag from the ASHE reference period to publication is usually six to seven months.

Like AWE, ASHE only covers employees and therefore excludes the self-employed. The selection of personal characteristics is also more limited than some other sources of earnings. For example, disability, ethnicity, nationality, country of birth and qualifications are included in the LFS, but not in ASHE, making LFS the best source for earnings estimates of these characteristics. A further limitation is that there is no information on what individuals are doing when they are not present on the survey. Potentially they could be unemployed; have switched to self-employment; have exited the labour market for a period or their employer may not have responded to the survey.

There are known coverage issues with data on bonus and incentive payments relating to the reference period. Primarily, this is because the information is not always available to respondents at the time when they are required to provide the information to ONS.

ASHE estimates may be affected by non-response bias. This is likely to be a downward bias on earnings estimates since non-response is known to affect high-paying occupations more than low-paying occupations.

ASHE statistics are also subject to non-sampling bias. Jobs that are not registered on PAYE schemes are not surveyed. These jobs are known to be different from the PAYE population in the sense that they typically have low levels of pay. Consequently, ASHE estimates of average pay are likely to be biased upwards with respect to the actual average pay of the employee population.

Since 2000, discontinuities in the series exist in 2004, 2006 and 2011. ASHE replaced the New Earnings Survey in 2004, with subsequent methodological changes in 2005. There is further information on series discontinuities and survey changes later in the article.

4.2 AWE

4.2.1 Strengths

A key strength of MWSS (the source survey of AWE) is its frequency, which enables AWE estimates to be produced on a monthly basis, usually with a six to seven-week gap between the end of the reference period and the publication date. This timeliness and frequency is why AWE is ONS’ lead indicator of short-term changes in earnings. It is the AWE measure of average earnings that is most often compared with measures of inflation to calculate changes in real earnings.

Another strength of AWE is its ability to capture bonus payments. With the MWSS being conducted monthly it captures bonus payments in every month of the year, with March often being the main month in which bonuses are paid. An article Average Weekly Earnings – Bonus Payments in Great Britain is published on our website once a year.

Given its strength in capturing bonus payments, AWE is also used to supplement ASHE data in ONS estimates of public and private sector pay differentials. ASHE data are adjusted at an industry level, in line with the AWE measures of bonuses. This is crucial given bonus payments are one of the key differences in remuneration between the public and private sectors.

4.2.2 Limitations

The MWSS excludes businesses with fewer than 20 employees to limit costs and respondent burden. Employment figures for these businesses are taken from the Inter-Departmental Business Register (IDBR), while earnings are estimated using the relationship between earnings of large and small employers derived from ASHE, which does cover small businesses.

The MWSS does not collect any information on individual employees’ characteristics and as such does not allow analysis beyond sector and industry. The composition effect captured by AWE refers only to changes between industries and therefore not compositional changes within other characteristics, such as occupation and age, or changes within the same industry.

It should also be highlighted that AWE does not differentiate between full-time and part-time workers, and so a relative increase in the prevalence of part-time working would indicate that average weekly pay was falling whereas average hourly pay may remain the same.

Further details about the methodology for both ASHE and AWE are provided in Annex D and are also available on the ASHE methodology and guidance and AWE methodology and guidanceweb pages.

Given that this article will later compare the Annual Survey of Hours and Earnings (ASHE) and Average Weekly Earnings (AWE) series, it is relevant to consider the survey changes over time that might impact the individual series.

ASHE

In 2004, ASHE replaced the New Earnings Survey (NES). Several changes were made to the survey, including how missing responses were estimated (particularly regarding whether employees’ earnings were affected by absence), the weighting of the results and the coverage of the survey. In 2005 there were further questionnaire changes; however data from 2004 were adjusted to account for these changes to avoid a break in the series.

Since the ASHE (and NES) series began, there have been a number of changes to coverage, methodology and classification conventions for the survey. These have resulted in a series of discontinuities where the data may not strictly be comparable between any given pair of years where a discontinuity exists. However, it is thought that when comparing over a longer time series the discontinuities have a minimal impact on long-term trends.

Aside from the discontinuity between the NES and ASHE surveys in 2004, other discontinuities in the ASHE series also exist in 2006 and 2011. For each of these discontinuities, two versions of results have been produced, enabling valid comparisons with series on either side of the discontinuity; 2006 data were produced on the same basis as 2005 and 2007 separately, to cover the questionnaire and sample methodology changes respectively, and 2011 data were produced on both Standard Occupational Classification (SOC) 2000 and SOC 2010 bases.

AWE

In 2010, AWE replaced the Average Earnings Index (AEI). Unlike the AEI, AWE reflects changes in the composition of the workforce and has a better method of handling non-response. AWE also explicitly adjusts for small businesses in the economy (using ASHE to calculate the relationship between wage growth of firms with over 20 employees compared with those with fewer than 20 employees) whereas the AEI assumed the pay growth of small businesses was the same as that of larger businesses.

When the relationship between wage growth of large and small employers was calculated, the AWE data were revised to include these figures back to July 2010. The data prior to this were re-estimated to account for the impact of the revisions, to avoid a discontinuity.

There are no discontinuities or breaks in the AWE series, due to the whole series being revised back to 2000 if any major changes to methodology occur. The only exception is where businesses were reclassified between the public and private sectors which can affect the long-term comparability of AWE’s public/private sector series. A timeline of survey and classification changes is provided in Annex E.

Although both Annual Survey of Hours and Earnings (ASHE) and Average Weekly Earnings (AWE) measure changes in average weekly earnings, as discussed in the previous sections, the headline measures are not immediately comparable. On the one hand, ASHE headlines on the median measure of central tendency, while AWE headlines on the mean, each of which can result in significant differences as a result of the structure of earnings. On the other hand, there exist many differences between the ASHE and AWE methodologies which also result in significant differences between the two estimates of earnings. Figure 1 highlights the differences between the two headline measures over time when indexed to 2005=100.

Note that indexing has been used to compare the differences in trends between the two series as the level of earnings vary significantly from each other, with the headline ASHE measure consistently estimating a higher level of pay compared with the headline AWE measure (this is largely due to ASHE headlining on full-time employees who have not had a loss of pay in the reference period, a distinction not made in the AWE headline measure – see Annex F). The year 2005 has been used as the base year of the index due to changes to the ASHE survey in 2005 (these changes are briefly described in the timeline of survey changes section). While the implemented changes were designed so that they did not have an effect on the published headline ASHE estimates, other non-headline measures were affected, including our comparisons later in the article. For this reason, the ASHE and AWE series are compared from 2005 onwards.

Comparing the two indexed series shows they follow each other fairly closely until 2011, where the ASHE growth trend drops further below the AWE series resulting in a divergence. The gap as a result of this divergence continues¹ up to and including 2016 (the most recent period for which ASHE data are available). Readers should be aware that the calculated AWE index is representative of single month year-on-year growth rates, which are more volatile than the three-month year-on-year headline AWE growth rates typically used.

In order for users to make more valid data comparisons between ASHE and AWE, a “like for like” measure of earnings is required. This is best done by adapting the ASHE data to the AWE measure because ASHE has a much greater number of variables than AWE and therefore it is easier to adapt ASHE to fit AWE, rather than the other way around. There is known under-coverage in ASHE bonuses so we will exclude bonuses from our comparisons as they may not give an accurate picture if they were included. Additionally, AWE regular pay (which excludes bonus payments) is the most “stripped down” measure, so this will be easier to compare with the ASHE series. We shall use the non-seasonally adjusted series for AWE regular pay as the data on ASHE is not seasonally adjusted.

The most comparable ASHE measure to AWE regular pay is mean gross weekly pay excluding bonuses, including employees whose earnings were affected by absence and employees on trainee and junior rates, at the Great Britain level. The headline measure for ASHE is median earnings; however we will use the mean rather than the median to be consistent with the AWE earnings measure.

The comparable ASHE measure, detailed above, also includes basic pay, shift premium payments, overtime pay, paid leave, maternity pay, sick pay and other pay². The AWE regular pay series that we will be comparing with this ASHE series also includes these elements (although it may not include all elements of other pay captured by ASHE). The comparable ASHE series is not a published series, but can be calculated using existing data. Due to the fact that ASHE is an annual survey, with the reference month in April each year, as with Figure 1, we will also only make data comparisons with April AWE single month data periods.

In summary, the following elements are used to calculate the comparable ASHE series and the AWE regular pay series (see Figure 2):

• Great Britain
• whole economy level
• all employees (i.e. not considering full-time and part-time separately)
• mean weekly pay
• non-seasonally adjusted
• AWE values for April each year
• the most recent (revised) ASHE data to be used where breaks exist in the series (i.e. in 2006 and 2011)
• ASHE to include employees whose earnings were affected by absence
• ASHE to include employees who are on trainee or junior rates

And the series will exclude:

• bonuses and incentive pay

Looking at the comparable ASHE series with AWE in Figure 2, the series now track each other more closely up to 2010 but there is still a notable divergence in 2011. This is followed by a return to general stability in the magnitude of difference between the two series before narrowing in 2016.

Notes for: Comparison between the ASHE and AWE series

1. The spike in AWE in 2013 is a result of the deferral of bonus payments, particularly in the financial sector, from March to April 2013. More information on the effects of this can be found at Average Weekly Earnings - Bonus payments in Great Britain, 2012/13

2. Other pay mostly consists of allowances, e.g. car allowances paid through payroll, on-call and standby allowances, clothing and laundry allowances, first aider or fire-fighter allowances, etc.

Some potential reasons for the divergence between the two series in 2011 are:

• Standard Occupational Classification (SOC) change for Annual Survey of Hours and Earnings (ASHE), moving from SOC 2000 to SOC 2010
• Standard Industrial Classification (SIC) change for Average Weekly Earnings (AWE), moving from SIC 2003 to SIC 2007
• implementation of a methodological change for AWE, to improve earnings estimates for small businesses
• replacement of the Average Earnings Index (AEI) by AWE in 2010
• reclassification of banks, further education and large companies between the public and private sectors, which affects the AWE estimates

Of the above, the move to SOC 2010 for ASHE estimates in 2011, carried out in order to better reflect the nature of jobs in the economy, is likely to be the largest driver of the divergence between the two series. This is because the occupational classification is used as part of the methodology by which individual jobs are allocated calibration weights. These weights determine the extent to which each job in the ASHE dataset influences overall estimates.

Overall, earnings estimates produced under SOC 2010 are lower than those produced under SOC 2000 (Figure 3), where the ASHE estimate produced under SOC 2000 for 2011 increases and thus the index increases, while the 2011 estimate on the SOC 2010 basis is actually lower than the 2010 estimate, resulting in a decrease in the index. Primarily, the lower levels of earnings under SOC 2010 is because many of the highly paid jobs have a smaller influence on domain estimates due to having lower weights than they did under SOC 2000. The weak growth for ASHE in 2011 is in contrast to AWE which showed strong growth in 2011; as SOC is not used in the compilation of the AWE estimate, the SOC change had no effect on the AWE series.

Figure 3 illustrates that the SOC transition is not the only driver for the divergence in 2011, with the other reasons listed above also suspected to contribute to the divergence although to a lesser, unquantifiable extent.

When changing the headline series to be comparable with each other, there is a notable narrowing of the divergence between the AWE and ASHE series in 2016. In 2016, the ASHE release saw the publication date brought forward by 3 to 4 weeks. We used this release date change to prioritise a number of process changes which were designed to increase processing speed by improving the targeting of records to be validated whilst minimising the impact on headline measures derived from ASHE. This means that in terms of the headline ASHE publication, the figures remain largely unaffected, but non-standard analyses (including the analysis presented in Figures 2 and 3) may exhibit increased volatility in 2016. Consequently, this affects the comparable series as the standard ASHE filters are not being used (employees whose earnings were affected by absence and employees who are on trainee or junior rates are filtered from headline ASHE results), which all users are advised to use and are used on all standard ASHE analyses.

Although Annual Survey of Hours and Earnings (ASHE) and Average Weekly Earnings (AWE) measure similar aspects of the labour market, they are two different measures of employee earnings that can exhibit notably distinct differences when compared against one another. To help understand why, this article has given an overview of the two sources, including what each output measures, their strengths and limitations and the ASHE and MWSS surveys’ sampling, processing and weighting methodology.

The article discusses how the changing composition of the workforce, an often misunderstood aspect of ASHE and AWE, can significantly impact the measured average earnings. Some of the compositional factors which affect estimates of average weekly earnings include changes in the number of part-time or full-time staff in the labour market, changes to the number of hours employees work and employees entering and exiting the workforce.

The article then moved on to explain the differences between headline ASHE and AWE measures and between an ASHE series on a “like for like” basis with the AWE regular pay series.

A notable divergence between the headline AWE series and both the headline ASHE and comparable ASHE series’ occurred in 2011, with the resultant gap remaining thereafter. It is suspected that the primary reason for this divergence is the implementation of SOC 2010 in ASHE, although other factors may have also had an impact. These include the revisions made to historic AWE data after the implementation of SIC 2007, the improvements made to the estimation of small businesses on AWE, and the reclassifications of businesses between the public and private sectors on AWE.

Overall, the article illustrates that a wide number of factors exist which are potentially responsible for the observed differences between the two measures. Users should keep these in mind when wishing to do comparisons between them. In summary, the two measures have moved in broadly similar ways in the recent past, but they differ reflecting their respective strengths and weaknesses. Each will therefore be better suited to some uses over others.

Background information

References

A Guide to Labour Market Statistics

A Guide to Sources of Data of Earnings and Income

ASHE Changes in 2005

Average Weekly Earnings, AEI and AWE: Weale Report

Changes to ASHE in 2007

Estimating Differences in Public and Private Sector Pay, 2012

QMI ASHE

QMI AWE

Understanding average earnings for the continuously employed

Compare and contrast

This table compares the ASHE and AWE headline measures.

Five employees who, in 2014, were paid as follows:

Here, the average (mean) weekly pay was £313.20.

Suppose employee A left the company and was not replaced. In 2015 there were 4 employees, paid as follows:

The average weekly pay in 2015 was £355.50; this is 14% higher than in 2014 even though everyone who was working in the company in 2015 worked the same hours and earned the same hourly pay as in 2014. This example illustrates that a reduction in the number of low paid, part-time employees can affect average earnings.

Now suppose employee D reduced their hours from 36 to 18 in 2016:

Comparing the earnings of those in continuous employment, the mean weekly pay in 2016 was £303.75, 3% below the 2015 weekly earnings. Despite the fact that none of the employees’ hourly rates changed, wages decreased. In this scenario, the composition of the workforce changed, due to an increase of part-time employees.

This table gives a brief overview of the ASHE and AWE methodology, in particular how they are sampled, processed and weighted.

Notes for: Overview of methodology

1. Further information relating to ASHE methodology may be found on the Annual Survey of Hours and Earnings methodology and guidance page of the ONS website.

Notes for Annex E: Timeline of survey and classification changes

1. A number of larger businesses have “special arrangements” in place with the ONS to provide their data electronically. For more information, see: Annual Survey of Hours and Earnings (ASHE) methodology and guidance