Public service productivity estimates: Education QMI

• approximately 80% of the output estimate is based on numbers of pupils in publicly-funded schools in the UK
• estimates of volume of publicly funded education inputs are made up of 3 components: labour, goods and services and capital services
• produced on a roughly annual cycle around 15 months after the end of the reference period
• classified as National Statistics.

We launched the UK Centre for the Measurement of Government Activity (UKCeMGA) in 2005 to improve the measurement and reporting of government output and productivity. We have carried out an extensive programme of development work and established a methodology for estimating productivity growth in publicly funded education. We now produce annual estimates for education productivity on a calendar year basis and the estimates have National Statistics designation.

The public service productivity estimates for Education contains output, inputs and productivity estimates for public service education in the UK. Estimates of education output include an adjustment for quality based on the GCSEs, and equivalent attainment levels, of school pupils aged 15 to 16 years old. Education inputs are broken down into growth of labour, goods and services and capital.

A report detailing the sources and methods for education has been published alongside a worked example of how the productivity index is calculated.

There are a range of users and data providers for these estimates, including the government, regulatory bodies, the media and general public, academia and international statistical bodies.

The UK Centre for the Measurement of Government Activity (UKCeMGA) was launched within Office for National Statistics (ONS) in 2005 to take forward the recommendations from the Atkinson Review on the measurement of government output and productivity. Between 2005 and 2010, UKCeMGA carried out an extensive programme of development work and established a methodology for estimating productivity growth in publicly-funded education. We now produce annual estimates for education productivity on a calendar year basis and the estimates have National Statistics designation.

The output measure of education is compiled using a Laspeyres cost-weighted volume index based on full-time equivalent numbers of students in various institutional settings. The general derivation of such an index is explained in Documentation on current methods used for national accounts (2008). The inputs measure of education is also a Laspeyres volume index, and the productivity index is calculated by dividing the output index by the inputs index.

A major component of the public service productivity estimates for education is that the volume of output is adjusted for changes in quality over time. For England, this quality-adjustment factor is based on Level 2 attainment, which measures the proportion of students attaining the threshold of five approved GCSE subjects or a Level 2 vocational qualification of equivalent size. This method replaces use of average point scores for England from 2008 onwards. Before 2008 GCSE average point scores are still used.

This methods change was introduced due to a structural break in the attainment series in 2012 to 2013. Evidence that suggested significant increases in the average point score for England between 2008 and 2009 and 2012 and 2013 can, in part, be attributed to increases in the number of non-GCSE examinations taken and are not fully reflective of changes in education quality. The selected Level 2 attainment statistics provide a more consistent historical time series, which will also be more resistant to future planned changes in the education system. These changes have led to a downward revision to the annual average growth rate of quality-adjusted education output for 2008 to 2012, from 4.0% to 3.6% per year. This has consequently reduced productivity growth over the same period.

Average point scores are used for Scotland and Wales. For more information on this method change, please see Methods changes in public service productivity estimates: education 2013.

Caution should be exercised when interpreting the change in the quality-adjusted output and productivity series in recent years due to changes in the examination system and performance tables in England, which have influenced both the number and type of examinations sat by some pupils. These changes may affect the comparability over time of the attainment data used for the quality adjustment, particularly in the period from 2012 to 2013.

This report contains the following sections:

• Output quality
• About the output
• How the output is created
• Validation and quality assurance
• Concepts and definitions
• Other information, relating to quality trade-offs and user needs
• Sources for further information or advice

We have developed Guidelines for measuring statistical quality; these are based upon the five European Statistical System (ESS) quality dimensions. This report addresses these quality dimensions and other important quality characteristics, which are:

• relevance
• timeliness and punctuality
• coherence and comparability
• accuracy
• output quality trade-offs
• assessment of user needs and perceptions
• accessibility and clarity

More information is provided about these quality dimensions in the following sections.

Relevance

(The degree to which statistical outputs meet users’ needs for both coverage and content.)

There are three different statistical outputs published in public service productivity estimates: education:

• a chain linked volume index of education output, adjusted for quality
• a chain linked volume index of education inputs
• a derived volume index for education productivity (output per unit of input)

The estimates of the volume of education output vary from the series presented in UK National Accounts as general government final consumption expenditure (GGFCE) on education. The national accounts do not include a quality adjustment factor for education output, whereas the productivity estimates include a quality adjustment factor based on annual GCSE results.

The volume of output measure in public service productivity estimates also includes students in further education for all ages in the UK. We classify funding for further education colleges as part of the non-profit institutions serving households (NPISH) sector and it is therefore not included in national accounts’ measures of GGFCE for education.

The estimates of volume of publicly-funded education inputs consist of three components: labour, goods and services, and capital services. Labour input for schools is measured directly using administrative records of staff resources from each of the countries in the UK. The value of other kinds of labour input from local government is included in the weight used in aggregating schools and other types of labour input, for example, from central government together.

Data on goods and services expenditure by local and central government are drawn from national accounts and suitable deflators matched to the expenditure to remove the effects of prices.

Data on capital services for education are taken from the experimental series, Volume Index of Capital Services produced by ONS. Productivity change in publicly-funded education is calculated by dividing the change in the volume of output by changes in the volume of input.

There are a range of users and data providers for these estimates. These include the responsible governments and departments of the countries of the UK, regulatory bodies, the media and general public, academia and international statistical bodies. Users are generally supportive of the estimates of productivity that are produced by ONS and provide positive feedback on their analytical value.

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.)

Public service productivity estimates: education 2013 includes measures of the productivity of education from 1996 to 2013 using academic year information available from 1995 and 1996 to 2013 and 2014 on pupil and student numbers, pupil attendance rates and education staff.

Associated expenditure (including 2013 and 2014) in financial years is included for output cost weights and inputs data. Other data are drawn from national accounts and Eurostat returns on a calendar year basis, and includes data up to 2013.

Some activity data on pupil numbers are available around 8 months after the end of the calendar year reference period. School workforce data are generally available on a similar timetable.

Financial information is generally available around 12 months after the end of the reference period and some national accounts data are available in the annual blue book, around 9 to 10 months after the end of the reference period. Education productivity figures are therefore released around 15 months after the end of the calendar year reference period, using minimal number of forecast data.

Provisional dates are published on the release calendar. Actual publication dates are finalised at least 1 month ahead of publication. Good practice is to provide 12 months advance notice of release dates. In the unlikely event of a change to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Official Statistics.

Education output estimates

Education output measures are classed as A measures according to the Eurostat Handbook on price and volume measures in national accounts (2001). This handbook outlines in detail how measures of price and volume are categorised and the criteria or level of manipulation required for each method category (A, B or C). The basic principles of each method are as follows:

• A methods: methods that approximate the ideal as closely as possible
• B methods: methods are acceptable approximations, but are further away from the ideal
• C methods: methods of approximation that are too far away to be acceptable

Coverage that is directly funded by government in the article matches the national accounts definition of publicly-funded education, and comprises:

• publicly-funded places at private, voluntary and independent (PVI) nursery schools
• nursery classes in primary schools
• primary and secondary schools (including school sixth forms) – local authority maintained and for England only schools that are funded by central government such as academies, free schools and studio schools
• special schools
• city technology colleges (CTCs)
• student numbers on Initial Teacher Training and health professional courses

Further education provision that is indirectly funded by government via the non-profit institutions serving households (NPISH) sector is also included in the coverage of the education output estimates.

Activity data

The activity data for schools are from census sources (for example, the Pupil Level Annual School Census collected by the Department for Education (DfE)) and are the most accurate sources available. The sources are detailed in a separate article Sources and methods for public service productivity estimates: education. Pupil numbers are adjusted for attendance and the data are disaggregated by school type.

Cost weights

To produce a measure of output, the growth rates of individual activities are weighted together according to their share of total education expenditure to form an aggregate education output index, according to the Laspeyres methodology as used in the national accounts. It is not always possible to match activity to exact expenditure so there may be some approximation regarding this. For example, health professional training costs weights are based on unit cost data, which are not available for each year under analysis; where possible, estimates of unit costs for missing years are made based on the trends in actual data. Work will continue to improve the cost weights.

Quality adjustment

The current schools adjustment for England uses Level 2 attainment, which measures the proportion of students attaining the threshold of five approved GCSE subjects or a Level 2 vocational qualification of equivalent size. This method replaces use of average point scores from 2008 onwards. Before 2008, GCSE average point scores are still used.

The new quality adjustment for England will continue to be applied for Northern Ireland while a suitable data source on attainment is investigated. Average point scores for Scotland are no longer available as a result of major changes to the education system and identification of a suitable alternative is not yet plausible while changes are ongoing. As the average point score for Scotland has followed a consistent trend over time and Scotland makes up a small proportion of UK education expenditure, as a short-term solution the quality adjustment for Scotland has been forecasted for 2013 and 2014 using a 5-year geometric average over the period 2007 and 2008 to 2012 and 2013.

Average point scores for GCSEs continue to be used for Wales where there have not been significant changes to the education system. Further information on the quality adjustment for schools output can be found in Methods changes in public service productivity estimates: education 2013.

Initial Teacher Training is adjusted for quality using the change in the proportion of final-year students who attain Qualified Teacher Status (QTS) in England. Data are available from 2001 and 2002 and these have been used to construct a quality adjustment that is applied to the whole of the UK.

Volume of education input estimates

Labour

Estimates of education labour consist of teaching and support staff in local authority-maintained schools (measured directly) plus local and central government administrative staff (measured indirectly). Previously, academy staff had not been included in direct labour measures due to lack of data, however, this is now available from 2010 onwards.

Direct measures of teaching and support staff are taken from school census data for each of the countries in the UK. We then calculate the change in the number of hours-adjusted full-time- equivalent staff for each category, and weight the changes together using expenditure shares.

A new teacher hours adjustment method was introduced in 2014. The methods change was to use the Labour Force Survey produced by Office for National Statistics (ONS) to assess teacher hours rather than use the previously available teacher ‘Workload Diary’ published by DfE, which was substantially changed in 2012.

Further details of the change can be found in the Methods change in public service productivity estimates: education 2013. School support staff figures are also taken from workforce censuses in each country and weighted by gross annual earnings from the Annual Survey of Hours and Earnings (ASHE) produced by ONS.

Central government staff are measured indirectly using expenditure data from national accounts and European data returns, deflated by the “public administration” Average Weekly Earnings Index (AWE).

Goods and services (procurement)

Education goods and services are measured indirectly using comprehensive annual current price expenditure data provided by government departments to HM Treasury and figures from national accounts. This expenditure is deflated using sub-components of consumer, producer and service sector price indices that most closely match individual categories of goods and services expenditure. These deflators are on a Standard Industrial Classification 2007 (SIC 2007) consistent basis from 2010.

Capital services

Estimates of education capital services are taken from Volume Index of Capital Services produced as experimental statistics by ONS on an annual basis, and represent the positive contribution of capital to the production of output in a given period, rather than the traditional measure of capital depreciation, which estimates the amount of capital used up in the production process in a given period. The estimates can be volatile and are therefore smoothed over time.

Public service productivity estimates: education 2013 contains an analysis of trends in productivity, output and inputs including context and possible causes. The construction of the output measure is explained including the contribution from and relative expenditure on each education setting and the adjustments for attendance and attainment. Inputs are analysed by component in volume terms. A revisions analysis is also included alongside background

Accuracy

(The degree of closeness between an estimate and the true value.)

Both output and inputs series are constructed using a variety of administrative data. Data accuracy of the derived series therefore depends on the accuracy of the source data. Unless ONS has introduced substantial methodological changes, the main source of revisions to the education productivity estimates will be due to changes in source data and expenditure weights.

There are no directly comparable figures produced by other organisations to provide a contrast to the estimates produced by Office for National Statistics (ONS). This is because the public service productivity estimates published by ONS use an expenditure framework, that is, the government as a purchaser of services on behalf of citizens, rather than as a producer. The figures therefore give a measure of the technical efficiency of the government’s provision of services, whether produced by state-controlled organisations, or purchased from the private sector. Traditional methods of productivity estimation use a production or supply framework and so are not directly comparable to these statistics for reasons of:

• coverage
• measurement of output differences
• quality-adjustment

A more detailed explanation of the comparability of ONS public service productivity estimates is given in Comparing public service productivity estimates.

ONS converts source data from financial and academic year to calendar year and aggregates results to a UK level, which makes it harder to make comparisons at a country level.

It is difficult for ONS to provide a confidence interval around its estimates given the multiple sources of data on which the estimates are based. The ONS collates triangulation evidence from independent sources, which provide additional context to inform the interpretation of the education productivity statistics.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)

The article combines output of education for each of the four countries of the UK from 1996. The output measure for education is mainly driven by the demographics of the four countries. The time series for each component is consistent over the time period. However, in some cases where data were not available on the exact definition for a given country, they were estimated from alternative sources.

Education inputs are measured at a UK level but draw on data from each of the four countries of the UK, particularly for direct measures of labour input. Estimates calculated from 2006 are consistent within each country and are coherent across countries of the UK.

Caution should be exercised when interpreting the change in the quality adjusted output and productivity series in recent years due to changes in the examination system and performance tables in England, which have influenced both the number and type of examinations sat by some pupils. These changes may affect the comparability over time of the attainment data used for the quality adjustment, particularly in the period from 2012 to 2013.

(Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output.)

Office for National Statistics (ONS) analysis is at the international forefront of measuring productivity in the public services. Measurement of output follows guidance in the System of National Accounts (SNA) 1993 and subsequent SNA 2008, and the European System of Accounts (ESA) 1995 and ESA 2010.

Measurement of outputs (including the need to measure change in quality), inputs and productivity follows the principles in the Atkinson Review.

Output quality trade-offs

(Trade-offs are the extent to which different dimensions of quality are balanced against each other.)

Public service productivity estimates for education are produced on a roughly annual cycle, around 15 months after the end of the reference period. At present the estimates are produced with little or no need to forecast data.

If publication dates were brought forward a greater number of data series would have to be forecast than at present. This would inevitably introduce greater uncertainty into the final productivity estimates than at present.

Assessment of user needs and perceptions

(The processes for finding out about uses and users, and their views on the statistical products.)

The most recent ONS consultation on statistics products was held between 2 September 2013 and 31 October 2013. Responses were used to inform the Office for National Statistics (ONS) work programme and effect on users if statistics included in the consultation were no longer to be produced. Estimates of education productivity were not included in this exercise and a previous consultation in 2010 found there was general support for public service productivity outputs.

Specific feedback on the value and effect of estimates of education productivity are received from holding regular user events and our education functional board, which meets twice a year. The education functional board consists of working level representatives of the education departments of the four countries in the UK. They provide information on policy and data issues that will affect ONS work, and are part of the quality assurance process for the final estimates.

The UK Statistics Authority assessment of our productivity statistics found a mixture of positive user views, but with room to improve the relevance and clarity of our releases.

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

Public service productivity estimates: education 2013 provides information on UK-level indices for output, input and productivity from 1996 to 2013. Further detail and relevant supporting data are also provided in Excel form as reference tables. Analysis includes average annual changes and identifies distinct periods of change and is illustrated with line graphs and bar charts with data tables. Background notes are provided for economic and education concepts and terminology.

Triangulation evidence is provided in the body of the article.

Sources and methods for public service productivity: education is available as well as changes to these methods in two methods change papers.

Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel.

We also offer users the option to download the narrative in PDF format. In some instances other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For more information, please contact us via email at sophie.danielis@ons.gov.uk.

More information regarding conditions of access to data is available:

• Terms and conditions (for data on the website)
• Copyright and reuse of published data
• Accessibility

Any forthcoming major changes in methodology for education productivity statistics will be published on the Guidance and Methodology pages.