Quality adjustment of public service public order and safety output: current method

This article describes the method Office for National Statistics (ONS) currently uses to quality adjust elements of public order and safety (POS) output. This measure is incorporated into estimates of total public service productivity only. It is not included within the UK National Accounts.

As stated in Principle A of the Atkinson Review (PDF, 1.08MB), “the measurement of government non-market output should, as far as possible, follow a procedure parallel to that adopted in the National Accounts for market output.” In the market sector economy, prices can be used to indicate changes in the quality of a good or service – an increase in the price of a good may equally reflect an improvement in quality as well as increases in cost. As a result, the change in quality is accounted for by the change in price differential. However, when measuring the output of public services this is much harder to do.

Some aspects of quality are captured directly by the activity measures used to estimate public service output. Other changes, such as a shift towards more expensive but more beneficial services, automatically raise output due to their increase in relative cost weights. These basic activity measures are common to productivity articles and national accounts, consisting of cost-weighted aggregates of activities and services performed.

However, within public services the price mechanism is considerably weaker or non-existent, as many goods and services are consumed free at the point of use. Accordingly, output for elements of public services, therefore, are adjusted to take account of changes in the quality, in line with the recommendations of the Atkinson Review (PDF, 1.08MB), reflecting improvements in outcomes that can be attributed directly to public service activity.

It is important to note that such quality adjustments are explicitly excluded from the measurement of output in the National Accounts Framework and are not part of the output series used in other Office for National Statistics (ONS) measures of productivity.

Introduced as part of Public service productivity estimates: total public service, UK 2015, when measuring the output and productivity of public order and safety (POS), we explicitly adjust the measure of output from the Criminal Justice System (CJS) to take account of changes in quality and improvements in associated outcomes. The basic activity measures, common to both public service productivity estimates: total public service, UK and national accounts, consist of cost-weighted aggregates of services provided (for example, prison population or number of cases by court type) which are paid for by the UK government. The quality adjustments we apply take into account some of the aspects of quality not already captured by the simple activity measure.

Within the public order and safety (POS) service area there are four main components: fire-protection, courts (which itself has five further sub-components), probation and prisons. The quality adjustments highlighted within this article are applied to the components as shown in Table 1 identified as forming part of the CJS. A quality adjustment is not applied to fire-protection or county courts services, which deliver civil cases. This is because these services are deemed to have different outcomes to the criminal justice elements of POS.

The criminal justice quality adjustment has four components. The first (recidivism adjustment) relates to achieving an overall outcome of the service and the remaining three relate to specific target outcomes for subcomponents of the CJS:

• recidivism adjustment

• prison safety adjustment

• custody escapes adjustment

• courts’ timeliness adjustment

Annexes A and B provide more technical information on the construction of each of the components of quality adjustment.

The following section outlines briefly the construction and application of the quality adjustments applied to the criminal justice element of public order and safety output measures, as part of the Public service productivity release. Due to the nature of the measures, the inverse is taken to represent changes in quality, typically with increases in the numbers reported representing a deterioration of outcomes and/or worsening of activity and service quality.

Further detail can be found in Annex A, Annex B and in Office for National Statistics’s (ONS’s) methodological article Quality adjustment of public service criminal justice system output: experimental method.

The recidivism adjustment

The recidivism adjustment is applied across all output associated with the Criminal Justice System (CJS). It approximates the effect the CJS has on reducing the volume and severity of further crimes being committed by those who have gone through it – this being an important social outcome for the system. The ONS measure works by adjusting the cost-weighted activity indices of the service areas identified in Table 1 by a severity-adjusted rate of recidivism.

This adjustment is composed of three parts, the first being the change in the number of proven re-offences committed by adults and juvenile offenders categorised between crime types. Annex A contains details on the crime type categories. An adjustment is made to adult offenders, to account for differences between cohort characteristics and their likelihood to re-offend. No such adjustment is made for juvenile offenders after 2005. The final adjustment made provides a weighting by which to aggregate together all re-offences. This weighting is based upon the relative severity of the re-offence and is derived from the ONS Crime Severity Score for England and Wales. More information on this source, as well as others used, can be found in Annex A.

The prisons safety adjustment

The prisons safety adjustment relates to the number of incidents of assaults, self-harm and deaths that occur in prison custody. We measure the number of incidents per 1,000 prisoners, which are grouped into “Severe”, “Less severe” and “Those resulting in a death”. These groups are subsequently weighted and aggregated together based on their relative cost. This is achieved by using the total cost to society of workplace injuries as a proxy, taken from the Health and Safety Executive.

The custody escapes adjustment

The escape adjustment relates to ensuring prisons fulfil the role of public protection and is applied to activities used to measure the output of the prison service.

The measure is based on changes in the difference between the number of escapes and a baseline of 0.05% of the England and Wales prison population – a historic target used by the Ministry of Justice. The purpose of this being that as the absolute number of escapes approaches zero, the relative change year-on-year would have a disproportionate effect on a non-baselined quality adjustment index.

The courts’ timeliness adjustment

The courts’ timeliness adjustment relates to the average time taken for criminal cases to be taken to completion, on the basis that the delivery of a sentence in a timely manner is favourable. However, there is currently no adjustment made to reflect whether there has been fair treatment of the suspect or victims or to allow the appropriate time for preparations of criminal cases with differing levels of severity or complexity.

For magistrate courts, the measure is based on the mean average time of charge and laying of information to completion. For Crown Courts, the measure captures the average waiting times experienced by all defendants and the mean time from main hearing to completion. As implemented, the measure accounts for changes in the average time taken to completion by criminal courts because increases in volume may reflect a worsening.

Combining the components

To then observe how the quality measures adjust the output of public order and safety (POS), each element is applied to the appropriate component of POS before aggregating together.

The recidivism adjustment is applied to all components of POS identified as being part of the criminal justice system — as set out in Table 2. Both the prisons safety and custody escapes adjustments are applied across all prisons’ output. The courts’ quality adjustment is applied to magistrate and Crown Court outputs. This means that some areas have multiple quality adjustments applied.

For each component, we calculate an overall growth factor to be applied to the basic activity index. For those areas where multiple adjustments are applied, the growth factors are applied on a weighted average basis. Table 2 outlines the weights used. To then aggregate together all the components of POS – including non-quality adjusted components – they are cost-weighted together to produce an aggregate index of POS quality adjusted output.

The recidivism adjustment, applied from 2000 onwards, measures the change in the number of re-offences committed per offender, weighted by the severity of the crimes committed. This is illustrated in Equation 1:

Equation 1

where

• α is the recidivism adjustment

• R is the total number of re-offences

• ß is the severity of offence committed

• Γ is the characteristic adjustment factor

• O is the number of offenders in a cohort

• t is period

• i is type of offence

Proven re-offences are categorised in one of eight categories of offence:

• Violence against the person

• Sexual offences

• Robbery

• Theft offences

• Criminal damage and arson

• Fraud

• Other crimes against society

• Summary

A proven re-offence is defined as any offence committed in a one-year follow-up period that resulted in a court conviction or caution in a one-year follow-up or a further six-month waiting period. Individual offenders are allocated to a quarterly cohort only once, based on their first eligible entry in a given quarter. For the estimates used after 2005, annual cohorts are formed by combining the four quarterly cohorts of the relevant year. An individual offender, therefore, can appear up to four times within a single annual cohort. As a result, there is a higher proportion and greater weight given to prolific offenders.

Prior to 2005, estimates used are reported as 12-month offender cohorts meaning that an individual offender could appear only once within a calendar year. Further details, definitions and explanation on this can be found in the Ministry of Justice’s Guide to Proven Reoffending Statistics (PDF, 1.11MB) and Response to consultation on changes to proven reoffending statistics.

As proven recidivism is related to the characteristics of offenders, the actual rate of proven re-offending will depend, in part, on the characteristics of offenders entering the system. To therefore account for this, the re-offending rate for adults is adjusted by their respective Offender Group Reconviction Scale (OGRS4/G) from 2005 onwards. The OGRS4/G is based on a well-established, peer-reviewed methodology for assessing and presenting re-offending risk.

The ORGS4/G uses age, gender and criminal history to assess and score the re-offending risk of a given group of offenders. These scores are then used to compare the relative likelihood of reoffending either over time or between different groups. The OGRS4/G model is specifically developed for adult offenders and is not applied to juvenile offenders, although an alternative adjustment — the Youth Offender Group Reconviction Scale (YOGRS) — is currently being developed. We will look to implement this, if possible, in future work. Again, further details, definitions and explanation on this can be found in the Ministry of Justice’s Guide to Proven Reoffending Statistics (PDF, 1.11MB). Prior to 2005, a similar model is used to estimate a characteristic adjustment factor but covers both adult and juvenile offenders. Further details on this model can be found in the Ministry of Justice’s 2013 Proven Re-offending Statistics: Definitions and Measurement (PDF, 324KB).

To further account for changes in the impact of recidivism on society, an adjustment is made to reflect changes in the severity of the re-offences. Severity of offences being committed is an important component of the social experience. It can, therefore, be argued that an increase in recidivism may not reflect a worsening of outcomes if, simultaneously, there is a fall in the severity of re-offences being committed.

To reflect this, a relative severity weight is derived for each group, taken from the Office for National Statistics’s (ONS’s) Crime Severity Score for England and Wales. These weights illustrate the relative offence severity as the ratio between offence type proportion of overall crime severity and the relative offence rate by offence type of total offence rate per 1,000 population. Table 3 shows these implied severity weights for the respective offence types.

In applying these weights, there are three points to note.

Firstly, that the proportional composition of severity re-offences committed matches the severity of total offences committed. Therefore, assumes there is no difference in composition between the two groups.

Secondly, summary crimes — which cover less serious offences such as TV licence evasion and driving while disqualified — are not separately captured by the Crime Severity Score and are, therefore, given an arbitrary weight half that of the least serious offence group.

Finally, it should be noted some of these categories cover a wide spectrum of offences, from relatively minor to severe, such as violence against the person, which can range from minor affray offences to murder. These are weighted by volume to provide the implied severity. It is for this reason (the large number of relatively minor offences) that violence against the person scores a low weighting when compared with robbery or sexual offences.

In the form given so far, the adjustment treats the Criminal Justice System (CJS) as one interlinking system, of which the primary outcome is to reduce the rate and severity of recidivism by providing appropriate disposals and rehabilitation services. However, it can be argued that the associated sub-components may have specific target outcomes, in addition to reducing recidivism.

Prison safety adjustment

The prisons safety measure, applied throughout the series, uses summary statistics on the safety in custody experienced by prisons in England and Wales, reflecting that safety of prisons is an important component of the quality in the activity and services provided, as set out in the Prison Safety and Reform White Paper.

To do this, reported incidents are categorised into three groups: “Severe”, “Less severe” and “Those resulting in a death”. The first group includes serious assaults and self-harm incidents requiring hospital treatment. The remaining assaults and self-harm incidents are summed to create the second group. Data on self-inflicted deaths and homicides forms the third group. All three series are then converted to a per prisoner population basis.

These groups are subsequently aggregated together dependent on their relative cost and severity. This is achieved by using the total cost to society of workplace injuries as a proxy, taken from the Health and Safety Executive.

Custody escapes adjustment

The custody escapes adjustment measure, applied throughout the series, uses the aggregated number of escapes from all prisons, prisons’ HM Prison Service escorts and contractor escort areas. However, rather than observe changes in quality as the change in the absolute number of escapes, the quality metric captures the change in the difference between the absolute number of escapes and a baseline of 0.5% of the prison population — a former Ministry of Justice key performance indicator (KPI).

By placing the metric relative to a baseline number, we avoid issues as a result by simply observing relative change, which gives higher weight to similar changes in absolute terms when initial levels are lower.

Courts’ timeliness adjustment

The final adjustment we make to the output of the Criminal Justice System, applied from 2011 onwards, is for average length of courts’ cases from charge or laying of information to completion. Whilst there are various possible ways of treating and aggregating average time taken, the measure adopted for magistrate courts aggregates the mean number of days taken from charge or laying of information to first listing and first listing to completion for England and Wales as reported in the Ministry of Justice’s Criminal court statistics quarterly.

For the Crown Court, the measure aggregates the average waiting time (time between date of case being sent to the Crown Court and the start of the substantive hearing) for defendants dealt with in trial (triable-either-way) cases and the mean number of days from main hearing date to completion for England and Wales. Again these are reported in the Ministry of Justice’s Criminal court statistics quarterly. The purpose of these being to provide a more complete coverage of defendants’, victims’ and witnesses’ experiences.