Quality assurance of administrative data used in Private Rental Housing statistics

The Index of Private Housing Rental Prices (IPHRP) is a monthly release that publishes private rental price indices for Great Britain, its component countries and English regions. The index is calculated using rental data collected by rent officers operating for the Valuation Office Agency, Scottish Government and Welsh Government.

Incorporating so many different data sources into any statistic involves a certain degree of risk. Administrative data, in particular, may be collected and compiled by third parties outside the Code of Practice for official statistics.

To further assure ourselves and users of the quality of our statistics, we have undertaken a thorough quality assessment of these data sources. This assessment is a continuous process and we will publish updates periodically.

We have followed the Quality Assurance of Administrative Data (QAAD) toolkit, as described by the Office for Statistics Regulation (OSR). Using the toolkit, we established the level of assurance we are seeking (or “benchmark”) for each source. The assurance levels are set as either “basic”, “enhanced” or “comprehensive”, depending on:

• the risk of quality concerns for that source, based on various factors, such as the source’s weight in the headline index, the complexity of the data source, contractual and communication arrangements currently in place, and other important considerations

• the public interest profile of the item which is being measured, and its contribution to the headline index

Through engagement with our suppliers, we have assessed the assurance level that we have currently achieved by considering:

• the operational context of the data; why and how it is collected

• the communication and agreements in place between ourselves and the supplier; the quality assurance procedures undertaken by the supplier

• the quality assurance procedures undertaken by us

Table 1 summarises the quality assurance benchmarks that were set, and the assurance levels that we have assessed each source at during this assessment.

As a result of this assessment, we have put in place an action plan to improve our quality assurance in some areas.

Data for the Valuation Office Agency (VOA) require a comprehensive level of assurance; however, we do not currently have access to the microdata, which limits our ability to quality assure the data; we have mitigated the risks involved by working with VOA to put in place aggregation methodologies, processes and quality assurance procedures and are requesting access to their data through powers of the Digital Economy Act.

Ministry of Housing Communities and Local Government (MHCLG) data require a comprehensive level of assurance; however, the data are constructed from a number of data sources, which have not necessarily been comprehensively quality assured. We are working with MHCLG to understand their departmental approach to the development of Quality Assurance of Administrative Data (QAAD) material.

We will continue to engage with our data suppliers to better understand any quality concerns that may arise and to raise their understanding of how their data are used in the construction of the Index of Private Housing Rental prices. We aim to publish an update to this QAAD at the beginning of 2019.

Our assessment of data sources is carried out in accordance with the Office for Statistics Regulation’s Quality Assurance of Administrative Data (QAAD) toolkit. We are striving for a proportionate approach to assessing the required level of quality assurance for the varied data sources used in the compilation of the Index of Private Housing Rental Prices (IPHRP). We seek to highlight and address the shortcomings that we have identified and reassure users that the quality of the source data is monitored and fit for purpose.

In this paper, we set out the steps we have taken to quality assure our data and our assessment of each source. In section 3, we outline our approach to assessing our data sources. In section 4, we discuss the assurance levels we are seeking for each data source, and the resulting assessment. Finally, in section 5, we detail our next steps towards achieving full assurance.

This publication is part of an ongoing process of dialogue with our suppliers, to increase our understanding of any quality concerns in the source data, and to raise awareness of how it is utilised. Through this document, we aim to provide information and assurance to users that the sources used to construct the IPHRP is sufficient for the purposes for which it is used. We will, therefore, review this document every two years. For more information on our IPHRP measure please see our guidance document about the IPHRP.

We have conducted our assessment of data sources used in the Index of Private Housing Rental Prices (IPHRP )using the Office for Statistics Regulation’s Quality Assurance of Administrative Data (QAAD) toolkit. We took the following steps for each data source:

• establish the risk of quality concerns with the data

• establish the level of public interest in the item that the data are being used to measure determine benchmark quality assurance levels, based on the risk and public interest

• contact the suppliers of administrative data to understand their own practices and approach to quality assurance; generally, this consists of the following steps:

  • send out questions to our data suppliers requesting information on their quality assurance procedures conduct follow-up meetings or discussions with our data suppliers to request further information and clarification
  • maintain ongoing dialogues with data suppliers to develop a better understanding of any quality issues in the data, and raise awareness of how the source data are used

• review our own quality assurance and validation procedures and processes

• conduct an assessment of each data source using the four practice areas of the Quality Assurance of Administrative Data toolkit:

  • operational context and data collection communication with data suppliers
  • quality assurance procedures of the data supplier quality assurance procedures of producer

• determine an overall quality assurance level based on our assessment

• if this assurance level does not match the benchmark assurance level, then put steps in place to work towards meeting the required assurance level

• review the quality assurance on an ongoing basis; we will publish a QAAD update every two years

3.1 Setting the benchmarks

In accordance with the QAAD toolkit, we have sought assurance for each data source based on the risk of quality concerns associated with that data source, and the public interest in the particular item being measured by that data source.

We considered a high, medium or low risk of data quality concerns based on:

• the importance of the data source in calculating IPHRP, what would we do if we didn’t have this data

• the complexity of the data source; for example, whether it is compiled from a number of different sources

• the existing contractual and communication arrangements currently in place

• other considerations, such as any existing published information on data collection, methodology or quality assurance, or mitigation of high-risk factors with the data

We considered a high, medium or low public interest profile based on:

• the level of media or user interest in the IPHRP or its sub-components

• the economic or political importance of the IPHRP

• any additional scrutiny from commentators, based on particular concerns about the data

Together the risk of quality concerns and public interest profile are combined to set an overall assurance level that is required for a particular source. This assessment is based on the following matrix, as provided by UK Statistics Authority (Table 2).

3.2 Quality Assurance of Administrative Data practice areas

We have aimed to assess the quality of each data source based on four broad practice areas. These relate to the quality assurance of official statistics and the administrative data used to produce them: our knowledge of the operational context in which the data are recorded, building good communication links with our data suppliers, an understanding of our suppliers’ quality processes and standards, and the quality processes and standards that we apply. This is in line with the Office for Statistics Regulations expectations for quality assurance of data sources. The full assessments for each data source can be found in Annex A. Table 3 provides a breakdown of these practice areas.

In this section we summarise our data sources, and consider the assurance level that we are seeking (or “benchmark”) for these. This is provided in Table 4:

The same rental data used in Index of Private Housing Rental Prices (IPHRP) is also used to construct the owner-occupiers housing component of Consumer Prices Index, including owner occupiers’ Housing costs (CPIH) under the rental equivalence approach. Within Annex A of the CPIH Quality assurance of administrative data can be found the detailed quality assurance practices applied to the Valuation Office Agency, Scottish Government, Welsh Government and Ministry of Housing Communities and Local Government data.

In the previous sections, we have considered quality assurance for all data sources in our Index of Private Housing Rental Prices. Of the data sources we investigated, there are some that need further work to reach the level of assurance we are seeking.

To address these shortcomings, we will carry out further steps to improve our quality assurance. All outstanding actions are summarised in Table 5 with details on what actions we intend to take to rectify them.

This version of the Index of Private Housing Rental Prices (IPHRP) Quality Assurance of Administrative Data (QAAD) is intended to act as a progress update. Over the next few months, we intend to continue engaging with our data suppliers and, where appropriate, put in place firmer ongoing communications mechanisms and data delivery agreements. Importantly, this QAAD is not intended to serve as a final record of quality assurance. We view supplier engagement and feedback as an ongoing process, which we will continue to follow. We, therefore, intend to publish a review to this QAAD every two years.