Valuation Office Agency property attribute data: quality assurance of administrative data used in Census 2021

The Office for National Statistics (ONS) has applied the Quality Assurance of Administrative Data (QAAD) Toolkit to the Valuation Office Agency (VOA) property attribute data.

The VOA data provide property characteristics information for domestic properties in England and Wales, with a good degree of accuracy for statistical purposes.

The ONS is investigating how VOA data could be used to directly, and indirectly, supplement Census 2021. Primarily, the ONS is committed to replacing the number of rooms census question using this data source.

VOA make every effort to collect accurate and up-to-date data and maintain their accuracy, but there are a number of limitations when used for statistical purposes. The source has limited ongoing audit and there is potential for distortive effects (because of data updates being primarily linked to the sale of properties). The effect of these issues will vary by geography and type of property.

Our research linked the VOA data to 2011 Census responses and found a high level of coverage, low levels of missing variables and good agreement with census and survey data (see Section 7). We also found that the probability of missing values for VOA number of rooms can be predicted by the values of the census household responses (see Section 8). Missingness occurs because of either a failure to link VOA data to the census or missing values in VOA data.

For Census 2021, measures are being put in place to take account of these statistical quality issues through statistical production methods such as edit and imputation.

VOA number of rooms for Census 2021 will provide a high-quality relative measure of size. For example, enabling the comparison of households across areas within the same time period.

The Valuation Office Agency (VOA) is an executive agency of Her Majesty's Revenue and Customs (HMRC). It has been responsible for banding properties for Council Tax in England and Wales since the tax was first introduced in 1993. The VOA property attribute data extract provided to the Office for National Statistics (ONS) currently contains approximately 26 million records. More information about the VOA data can be found in the source overview.

This report aims to provide information and assurance to users that the VOA property attribute data are monitored and fit for the purpose of producing statistical outputs. This report is part of an ongoing dialogue with the VOA, to understand and address the limitations of the data collected by the VOA for our statistical purposes.

The Quality Assurance of Administrative Data (QAAD) Toolkit explains the required level of quality assurance based on the data quality risk and public interest. Based on this, Section 4 sets out the level of quality assurance that is required for each of the uses of VOA property attribute data within Census 2021.

The toolkit also describes four practice areas associated with data quality assurance, which are summarised in this report:

• Practice area 1: operational context and administrative data collection (Section 5)
• Practice area 2: communication with data supply partners (Section 6)
• Practice area 3: quality assurance principles, standards and checks applied to data supplies (Section 7)
• Practice area 4: producers’ quality assurance investigation and documentation (Section 8)

The Office for National Statistics (ONS) is investigating how Valuation Office Agency (VOA) property attribute data could be used to directly, and indirectly, supplement Census 2021. This section covers the confirmed uses of VOA data in Census 2021.

Census output on VOA number of rooms

The ONS are committed to replace the number of rooms census question using VOA data as stated in the Census 2021 White Paper (PDF, 967KB). To achieve this, VOA data on number of rooms will be linked to Census 2021 responses.

The assessment of data sources is carried out in accordance with the Office for Statistics Regulation’s Quality Assurance of Administrative Data (QAAD) Toolkit. This strives to give a proportionate approach to assessing the level of quality assurance that will be required of a dataset. The assessment of the assurance level is in turn based on a combination of assessments of data quality risk and public interest. The toolkit sets out the level of assurances required as follows:

Notes:

1. Code of Practice for Statistics

The following subsection sets out the level of quality assurance that is required for each use of Valuation Office Agency (VOA) property attribute data.

Where the Office for National Statistics (ONS) uses the same data source for a number of different purposes it is normally more efficient to undertake some of the quality assurance only once. Each separate use may have a different level of assurance required so the highest level of quality assurance required will be applied for all uses in Census 2021.

Census 2021 statistical output on VOA number of rooms

For the purposes of using VOA data to produce a Census 2021 statistical output on number of rooms, there is a high risk of quality concern and a medium public interest so a comprehensive assurance (A3) will be applied.

The level of risk of data quality concerns is high because:

• VOA data have multiple sources such as information submitted from different councils, the planning portal, building developers, tenants and inspections
• property data are manually input by VOA
• VOA carry out quality checks immediately after the data are input; there are no audit processes or procedure to check the data unless a challenge is raised by either the taxpayer or a VOA List Officer
• VOA caseworkers have targets for the number of properties to input or update a week

The level of public interest is medium because:

• the VOA data will be used to produce the Census 2021 output on number of rooms
• VOA number of rooms will provide a relative measure of size enabling the comparison of households across areas within the same time period; this information can be used to estimate overcrowding and to derive the Carstairs Index and Indices of Multiple Deprivation (IMD), these are used across a wide area of policy and decision-making
• the statistical census outputs usually get moderate media coverage and are of moderate economic and political sensitivity

Therefore, a comprehensive assurance (A3) will be applied to the VOA data. The following sections outline the considerations taken in order to provide this assurance that VOA data are of sufficient quality to produce the Census 2021 output on number of rooms.

Organisations collecting the data

Valuation Office Agency (VOA) collate data from multiple sources including billing authorities, taxpayers and online resources. Data can be collected in person by either the billing authorities¹ or VOA by visiting properties and carrying out internal and external inspections. Inspections are less common now that VOA can obtain the information in other ways like talking to developers or using online resources.

Original purpose for data collection

VOA capture data about properties for Council Tax banding purposes. They capture property attributes including property type, number of rooms, number of bedrooms and floor space as well as several other attributes to do this.

Legal basis for data collection

It is a requirement of VOA to maintain accurate valuation lists for Council Tax under the Local Government Finance Act 1992. VOA must also update information that is held on the taxpayers when in contact with them because of the Data Protection Act 1998 requirement. VOA makes every effort to comply and ensure the data are accurate.

Data sources and collection

Data are collected by billing authorities in England and Wales as part of the process of billing Council Tax and from other sources including the planning portal, building developers, taxpayers and inspections.

Historically the main data collection method was through inspection and was stored as paper records. The data were transferred from the paper records to electronic storage as the central database in the early 2000s in preparation for revaluation in England and Wales². The central database is where all the information collected is now stored and specific data can be extracted and updated.

VOA now use the following methods to collect and update data:

• confirmation by the taxpayer of the data already held; agents acting for a taxpayer are asked to complete a questionnaire, which is then compared with internal records
• third party sources: information that is available to the public, for example, local authority planning records on the planning portal and information available from local authorities’ Finance Department building control records
• external inspection of the property where the relevant details visible from the road can be confirmed; for example, seeing the property type or the presence or absence of an extension
• internal inspections of the property where attributes can only be confirmed internally or a re-measurement is required

Triggers to collect new data

If a new property is built, the attribute data are collected and entered into the database as a new entry. Property attribute data are only updated where information comes to VOA’s attention that a valuation list entry might be inaccurate. There is a reliance on billing authorities to inform VOA of changes to properties as records are not updated unless a challenge is raised or new information is found about the property.

A challenge (or band review) can be raised by either the taxpayer or a VOA List Officer and requires evidence that a Council Tax band may be inaccurate. The VOA will investigate and, if necessary, change the band.

As set out in legislation, for the first six months of first moving into a property, the taxpayer can request a formal review of the Council Tax band (that is, submit a proposal). The VOA will review the band and issue a decision. If the taxpayer is not satisfied with the decision, they have the right to appeal to an independent Valuation Tribunal (VT).

Property attributes may also be updated by VOA when the council receive new information about a property or if planning permission is applied for by the taxpayer (see Section 7). In this case VOA would investigate the property using the appropriate means and ensure that all information held is up-to-date.

If a VOA caseworker comes across new information about a property they are not actively investigating, for example, a property on the same road, then the details will be updated for that property too.

If changes to a property are found, a structural alteration report is sent to VOA and an improvement indicator is put on that property. If the change results in a split, merge, change of use (that is, no longer domestic) or a new build, the change is actioned once the report has cleared. For other changes to a property, the change is not actioned until the property is sold. If the change to the property results in a band change, that is actioned at this point too.

When a property is known to be demolished, the record will be removed from the Council Tax valuation list.

Potential sources of bias and error in the administrative data

In principle, VOA’s statutory function is to maintain the accuracy of the valuation list – not the property attribute data. However, in effect the list and the underlying data are subject to continual review from the challenge process and at the very least any time a property changes ownership.

However, there is no regular review of property data without a challenge or new information about the property being found by the billing authority or a VOA List Officer. Changes to the property can occur without a trigger, for example, homeowners can change internal partition walls without the need for planning permissions or unlawfully make structural changes to their property without planning permission.

Without a trigger to investigate a property, errors originating from the original data collection or because of changes to the property, which have not triggered an investigation, may not get corrected. Given the maturity of the list and the number of house sales, the vast majority of properties will have been reviewed at least once and probably on several occasions.

Because updates to the data are primarily linked to the sale of properties, there is the potential for distortive effects, which cannot be mitigated fully. The effect of these issues will vary by geography and type of property.

Notes for: Practice area 1: operational context and administrative data collection

1. Billing authorities are local councils that directly bill Council Tax in their local area. To ensure that residents only receive one Council Tax bill, every local area has one council that acts as the billing authority. In two-tier areas the billing authority is the District Council.

2. The Welsh revaluation took place (based on 2003 values) but England’s was cancelled.

Supply agreement

Initial meetings between the Office for National Statistics (ONS) and Valuation Office Agency (VOA) took place to ensure the sharing of data was both reasonable and proportionate as required under the Commissioners for Revenue and Customs Act 2005. An Information Sharing Protocol (ISP) was drawn up in 2017 as the data sharing agreement between VOA and the ONS. It covers details such as the legal gateway and the data sharing procedures to be followed.

The legal gateway to share the data to the ONS was created through the Statistics and Registration Service Act 2007. The ONS received a snapshot of data in July 2016, which covered the period of 1 April 1993 to 1 July 2016. The ONS now receives monthly update files that are encrypted and transferred via MoveIT software.

In the ISP, the ONS agreed to store the data from VOA securely, with only those who have a business need able to access it for the purpose it was supplied for. The risks and effectiveness of mitigating measures are reviewed regularly.

More detailed information about the ONS's data strategy, policies and principles is available.

Regular communication with supplier

Statisticians from the ONS and VOA are in regular contact as part of the research process. To understand the data journey and assess how the data collection process may impact the data quality, statisticians from ONS’s Census Statistical Design Team visited the VOA Southampton office and met with VOA caseworkers who input the data. ONS publications that use VOA data are reviewed by VOA to ensure accuracy and consistency.

Approach to data collection and quality

As described in Section 5, the Valuation Office Agency (VOA) collect property attribute information for the purpose of providing domestic Council Tax valuation bands. The VOA maintains the valuation list, updating information about properties, adding new properties and removing properties where needed.

In principle, VOA will review property attributes (including the number of rooms variable) whenever new information comes to light, and an improvement indicator is added if a change has taken place. If the change was splitting the property, merging properties, new builds or a change of use (that is, to non-domestic), the property attribute data would be updated without a sale taking place. For other changes, the property attribute data are not updated until the case is cleared. This is triggered by notification that the property has been sold. If these changes would change the Council Tax banding, this change will be actioned once sold too.

The floor space and type of property are especially important in valuing property and assigning the Council Tax band, more so than the number of rooms. However, every effort is made to ensure the accuracy of all attribute data and multiple sources of information are used to do so.

Definition of a dwelling and identifiers within VOA data

The distinct units of observations for which the VOA holds and maintains information are dwellings. The definition of a dwelling is based on rateable occupation, which is the physical aspect of the owned or occupied property. However, the VOA Council Tax manual also explains that the Council Tax (Chargeable Dwellings) Order 1992 introduced the need to identify any “self-contained units” found within properties and treat them as separate dwellings.

From 1 April 1997, the definition of self-contained is “a building or part of a building which has been constructed or adapted for use as separate living accommodation.” Each dwelling has its own unique identifier assigned called Unique Address Reference Number (UARN).

Definition of a dwelling for Census 2021 and relation to VOA dwellings

Census 2021 is a census of households within dwellings. A dwelling is defined by its address on the census address frame and where a household resides. A household is defined as “one person living alone, or a group of people (not necessarily related) living at the same address who share cooking facilities and share a living room or sitting room or dining area”.

Census 2021 derives the address frame from AddressBase Premium which is constructed by GeoPlace Ltd using several sources including VOA data, Royal Mail Postal address file and Ordnance Survey Master Map. A property/address within AddressBase is given a Unique Property Reference Number (UPRN). UPRNs are hierarchical which means there are parent UPRNs to group child UPRNs, for example, flats within a block of flats.

All dwellings from VOA data (UARNs) are mapped to UPRNs so that VOA references can be included in AddressBase products. For a minority of records, GeoPlace are unable to uniquely map UARNs to the lowest level (child) UPRN. In that case a group of UARNs are assigned to a higher level (parent) UPRN. VOA monitors which UPRNs are assigned to UARNs and there is an ongoing effort to reduce the number of UARNs that cannot be assigned to a child UPRN.

As a result, the Office for National Statistics (ONS) is able to achieve high-quality linkage between Census 2021 address frame and VOA data. Every effort is made to link VOA records to the correct statistical unit, that is, the census dwelling for single households or a census household where dwelling contains multiple households.

Sources and mechanism for updating property attributes

The initial data collection and triggers to collect new data or update existing data are fully covered in Section 5. An important trigger for updating the property attributes for the valuation list is notifications in the form of Billing Authority reports. Billing Authority reports cover different types of changes including:

• new entries to the valuation lists
• splits or mergers of properties
• properties that were previously non-domestic being converted to domestic use
• structural alterations (material change)
• changes to composite properties (that is, increase or decrease in domestic use)
• demolition of part of a dwelling
• address changes
• taxpayer band reviews

VOA case workers are assigned to handle reports and collect the property information needed to update the valuation list. Where information received is incomplete or verification is required, this can be sought from a variety of sources such as Planning Portal, Estate Agents, property websites (for example, Rightmove), Google StreetView, Digital Mapping, physical inspections or communications with taxpayers. Information sources that can be accessed and utilised remotely are preferred and physical inspections are used when information is difficult to source by other means or requires additional verification.

In the case of new developments, attention is paid to check whether building plans and specifications change during the phases of development. For example, it is possible a developer may initially plan to build 20 houses of one design and 20 of another design, but the number of each design may be changed during the phases of the development. If a taxpayer makes changes to the property they are living at, VOA add an improvement indicator to the property data. Unless the change is a split, merge, new build or change of use, the property attributes data (and band if applicable) are not updated until the property is sold.

Quality checks applied by VOA

Following collection of data by VOA case workers, the data attributes are input manually into the VOA central operational database; 5% of records input are checked by managers weekly (also see “Oversight of quality checks by VOA”).

Specific checks are completed on the data for missing and incompatible codes. For example, the number of rooms should always be greater than the number of bedrooms (except studio or bedsit). Whenever VOA interact with a taxpayer, inspect a property or receive or research information about a property, all property attributes currently held by VOA should be checked and any missing attributes should be completed, if possible.

Searches have been run by VOA on missing and/or incompatible codes. In the latter case, exceptions lists were produced to pick up incompatible information, that is, bungalows with multiple floors and flats that are coded as semi-detached. If any incompatible codes are spotted whilst staff are carrying out their work, codes would be amended.

There are no validation checks built into the VOA central database but from August 2020, VOA will use a new estate file portal for new entries (CR03s). This will prevent incompatible codes from being entered.

The minimum standard is that on clearance of any report, proposal or appeal, the dwelling will have the eight primary property attributes completed:

• group (broadly a signal of architectural style)
• type (detached, semi-detached)
• age
• floor area (meaning size in metres squared)
• number of rooms
• number of bedrooms
• number of bathrooms
• number of floors

Further information on these property details can be found in VOA’s Property Details Guide.

Oversight of quality checks by VOA

Managers in VOA undertake weekly spot checks on new and changed entries to ensure sufficient quality of the data. A minimum of 5% of all new or amended entries and at least one case for every staff member must be checked by a manager once a week. Managers check all entries from new caseworkers for a period of time and reduce the amount of checks according to results. This means inexperienced staff have a far greater percentage of their work checked until they are fully trained.

Outliers and errors are flagged and signed off by managers before any amendments are made to the valuation list. Any errors found are resolved. Once processed, the information is passed back to the billing authorities to check the new valuation list entry is consistent with the information they hold.

The House Price Indices Production Team within the ONS produced a quality assurance of administrative data (QAAD) and shadowed VOA colleagues to ensure understanding of any data assurance practices. The Census Statistical Design Team also visited VOA colleagues to further improve ONS’s understanding. Quality checks performed by VOA on the data are predominantly at record level: either when the data are initially input or with property owners during any contact with them.

Given the ONS has access to record level VOA data, additional quality checks have been performed by the ONS on the VOA data. These have included checking for missing variables or inconsistencies and by matching against censuses and other surveys for comparison. For example, 2011 Census accommodation type has been compared with VOA property type and 86% of linked addresses were found to agree. This rose to 93% when more ambiguous matches were included. The full analysis is available.

Quality reports and audit of VOA data

VOA publishes statistics on the stock of properties by Council Tax band and attributes such as property type and age. It also publishes statistics on Council Tax challenges and changes. These statistics are based on information recorded in the VOA central database, which is the source of the data provided to the ONS. Each publication of VOA statistics includes methodology and quality information within the background information section.

There is no consistent national audit of VOA Council Tax data by an external body.

Strengths and limitations of VOA data

A strength of the data is that they are of high statistical quality, found in both the data published by VOA and in ONS feasibility research. The data have low levels of missing records in main variables and high coverage of England and Wales, which was clear when 96% of 2011 Census records linked to the VOA data. Some VOA data may be more accurate than census data as VOA data are input by a trained caseworker rather than self-reported by the public.

Limitations of the data are the definitional differences between the census questions and VOA variables. For example, the census included kitchens, utility rooms and conservatories in its number of rooms estimates, which the VOA data do not. Despite the definitional difference, nine respondents to the 2017 user consultation indicated that using VOA data would have a positive impact on their work compared with six respondents who indicated it would have a negative impact; 12 did not indicate either way.

VOA data are not routinely audited but both the quality checks at record level and statistical checks performed on the data before publication contribute towards mitigating this concern.

Implications for official statistics

There is a reliance on billing authorities to provide information or trigger an investigation of a property. Some billing authorities may be gaining the information by inspection more than others. Analysis by ONS comparing VOA and 2011 Census data on property type found agreement at local authority varied but was always above 69%. It was found that the local authorities with the lowest agreement rates are in rural areas. These areas are likely to contain a larger proportion of bigger properties.

It is also known from the 2011 Census Quality Survey (PDF, 1.42MB) that respondents in properties with four, five, six and seven rooms had the most difficulty in understanding the census definition of a room. Given these findings, a lower proportion of data collected by inspection is not thought to impact the quality of the data collected, especially as inspections are now not a main data collection method.

The number of rooms variable is one of the eight main property attributes that VOA caseworkers are required to input. Therefore, that variable is likely to be of high completeness and the weekly quality checks carried out by VOA managers further improve the quality of the data.

Common quality checks by the ONS

The VOA property attributes data will be used to provide the number of rooms for households in the Census 2021 (replacing a question asked in past censuses). A summary and methodology article have been published in preparation for this. As part of the Census White Paper Help Shape Our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB) commitments the ONS have also investigated whether statistics on property type and floor space from linked VOA and census data can be produced and provide benefit to users.

The production of statistics is distributed across different teams within the ONS so basic quality checks are done by a centralised team when data are received. Data are then delivered to research teams to produce statistical outputs. These teams will carry out more in-depth quality checks specific to their statistical output. Some of these checks can be found in previous publications on property type, floor space and VOA number of rooms.

Common processing by the ONS

Common processing is carried out by the ONS Data Engineering Team. An initial snapshot of VOA property attributes was taken in 2016 and supplied to the ONS. Since then, the ONS receives updates to these data from VOA every month in the form of six text files for changes, additions and deletions for England and Wales separately.

The VOA data files are processed to create current and historical data. The ONS add record start date, record end date (if applicable), a live record flag and a change event flag (new, old, updated, deleted). There is no additional cleaning, but records received are checked against those already held. Additions (new cases) are appended to the existing dataset and these records are checked to make sure they are whole new records rather than an update to an existing record. For deletions and changes it is implied that the record is already held.

Validation and quality checks by the ONS

Once common processing is complete, researchers check the data are in the expected format, and contain the number of expected records (compared with past data, and other sources such as 2011 Census households).

Comparisons of VOA number of rooms to 2011 Census

VOA number of rooms and bedrooms data were compared with the 2011 Census data to assess the quality of the 2016 VOA dataset. The 2016 VOA data were rolled back using the built age of property variable and removing records built after 2011. A high linkage rate of 96% achieved between the 2011 Census and VOA data indicates the coverage of the VOA data are good for statistical purposes. However, the aim within Census 2021 is to have no missing values of number of rooms for households.

There are definitional differences between the census number of rooms question and the VOA number of rooms variable. The 2011 Census includes kitchens, utility rooms and conservatories in its number of rooms estimates, which the VOA data do not. Since most properties have a kitchen, the number of rooms in the census data was generally higher than the corresponding number of rooms in the VOA data. Allowing for this definitional difference we found that the distributions of number of rooms are similar for 2011 Census and VOA data. More detail can be found in the number of rooms and bedrooms publication.

It is worth noting that the quality of the census responses for number of rooms was measured by the 2011 Census Quality Survey (PDF, 1.42MB) at 67%. The survey found that differences occurred because respondents had misunderstood the question. Most of these differences (93%) were within plus or minus one room.

Using VOA number of rooms for Census 2021 does imply a discontinuity with 2011 Census estimates (because of the definitional difference) that users need to be aware of. It will not be appropriate to measure change in number of rooms from 2011 to 2021 using this variable; instead, the census bedroom question can be used for comparisons over time.

Using the number of rooms in the VOA data for Census 2021 will provide a high-quality relative measure of size enabling the comparison of households across areas within the same time period.

Addressing missingness for the Census 2021 output on VOA number of rooms

Census 2021 will be the first time we are using administrative data linked to the census to produce a census statistical output. We need to ensure administrative data are suitable to undergo imputation to ensure that the quality of future census outputs are not impacted by households that are missing number of rooms data because of either a failure to link to VOA data or missing values in VOA data.

To test the methodology ahead of Census 2021, we have linked 2011 Census and VOA records (as at 2011) at address level. Our analysis demonstrates that it is feasible to predict missing VOA number of rooms using donor-based imputation from the 2011 Census household variables. Based on this analysis, it is the ONS view that linked VOA number of rooms data are suitable to undergo imputation in Census 2021 for England and Wales. More detail can be found in the summary and methodology article explaining our edit and imputation approach for addressing missingness in the linked census and VOA data.

We will consolidate our work by using 2019 Census Rehearsal data linked to equivalent VOA data. This will enable us to confirm that the quality of data linkage will continue to improve and it will remove time lags between census and VOA data. This will ensure we are operationally ready for Census 2021.

The Office for National Statistics (ONS) has applied a comprehensive assurance (A3) to the Valuation Office Agency (VOA) property attribute data using the Quality Assurance of Administrative Data (QAAD) Toolkit.

We found that VOA make every effort to collect accurate and up-to-date data and maintain their accuracy. Our research has verified that the data have a good degree of accuracy for statistical purposes, because of high coverage, low levels of missing variables and good agreement with census and survey data.

However, the VOA data have some limitations when used for statistical purposes. The source has limited ongoing audit and there is potential for distortive effects, because of data updates being linked to the sale of properties unless the property is split, merged, new or has a change of use. This means there may be an unknown time lag in the data, which is in line with other administrative data sources. The effect of these issues will vary by geography and type of property.

For the VOA number of rooms variable, there are definitional differences between the census number of rooms question and the VOA number of rooms variable. The census includes the kitchen, which the VOA data do not. Data users will need to be aware of this discontinuity and it will not be appropriate to measure change in number of rooms from 2011 to 2021 using this variable; instead, the census bedroom question can be used for comparisons over time.

For Census 2021, measures are being put in place to take account of statistical quality issues. For example, we will address missingness in data, because of a failure to link to VOA data or missing values in VOA data, using an imputation process, which can predict missing values using the census household responses.

Therefore, the ONS believes that VOA number of rooms for Census 2021 will provide a high-quality relative measure of size, for example, enabling the comparison of households across areas within the same time period.