US Armed Forces data: quality assurance of administrative data used in population statistics, Jan 2017

Each year data on numbers of US Armed Forces (USAF) located in the UK are requested from USAF to help estimate the number and distribution of armed forces personnel as part of the estimation of the usually resident population in local authorities in England and Wales. US Armed Forces are treated as a special population – termed the Foreign Armed Forces (FAF) – in the calculation of the estimates as the movements of military personnel are not captured by the usual data sources used to estimate international and internal migration.

The collected data, which assigns people to an Armed Forces base, is transformed to data by local authority of residence (as required for the population estimates) using data from the 2011 Census.

This document covers the processes, from data collection through to the population estimates produced by our Population Statistics Division. It identifies potential risks in data quality and accuracy as well as details of how those risks are mitigated.

This document does not aim to report on the whole of the mid-year population estimate processing or the quality assurance relating to the processing of the other components used in its production. Further information relating to the quality of the mid-year population estimates can be found in Population Estimates Quality Tools and in the Annual Mid-Year Population Estimates QMI on our website.

Population estimates are assessed as having a relatively high public interest profile. However, the US Armed Forces are a small part of the population. Any inaccuracies in the data would have a similarly small impact on the quality of the population estimates. We judge it unlikely that there would be a change in the information collected which would impact on the quality of the estimates. We have therefore judged that the appropriate level of assurance for this data source is A1.

The A1 rating means that a basic level of assurance is required for this source and this document will provide information to meet this level of assurance. If you feel that this document does not adequately provide this assurance then please contact pop.info@ons.gov.gsi.uk with your concerns.

The toolkit outlines 4 areas for assurance; the rest of this document will be split into these areas. The areas for assurance are:

• operational context and administrative data collection
• communication with data supply partners
• QA principles, standards and checks applied by data suppliers
• producer’s QA investigations and documentation

The data are initially collected when personnel arrive at their new duty location. These data monitor the provision of US Armed Forces (USAF) personnel in the UK. Each service member is required under USAF regulations to meet with the administrative staff of their new duty location on the first day. During this meeting the service member fills out multiple forms that include information on their dependants (spouses, children and others that reside with them). This information is then checked with the member to ensure its accuracy. Once all of the information has been checked it is then inputted into databases that track the information while the member and their family resides in the UK.

As there is no formal agreement to supply data in place, the data is requested and provided on the basis of past precedent.

Staff in Population Statistics Division (PSD) contacts US Armed Forces, via email, each year to request numbers of US Armed Forces by base and single year of age and sex. USAF provides the requested data as an Excel file.

This data is quality assured by staff in PSD (see section 5). The data are stored in a set-up folder for the reference year. This is on a shared hard disk drive with access restricted to PEU staff. If any problems with the data are identified, USAF is contacted for clarification and/or investigation.

3.1 Engagement with users

PSD continually engages with users to understand how well outputs meet their requirements. PSD’s user engagement activities include formal consultations on proposed changes to outputs, regular communication on plans through a quarterly newsletter, and external events open to all users. In addition, where evaluating changes to methods or sources has required specialist knowledge of local areas, PSD has organised Local Insight Reference Panels to elicit the views of relevant local authorities. From these activities, any issues relating to the sources and their fitness for the proposed use, will naturally come out. Issues restricted to one output will generally be addressed by the team responsible for that output while the Stakeholder Engagement team in PSD takes an overview of any issues with more general implications, and ensures that this is considered in development of outputs across the division. It should be noted that users are more likely to comment on the overall methodology and the effect that it has on the final statistics than on a contributory data source.

Any issues around the quality of the statistics are described in the Quality and Methodology Information report accompanying each output. Issues around specific administrative data sources used in producing the statistics are considered in Quality Assurance of Administrative Data reports such as this.

When changes are proposed to methods (including changes in data sources being used in producing statistics) the ONS Population Methodology and Statistical Infrastructure Division will assess the resultant methods prior to implementation to assure that they are of sufficient statistical quality to meet user needs and are an improvement on the previous method. An independent evaluation by academic experts may also be undertaken, should methodological changes be extensive. The methods are also subject to scrutiny by the UK Statistics Authority as part of the National Statistics accreditation programme under Principle 4 of the Code of Practice for Official Statistics (sound methods and assured quality).

The Responsible Statistician is named for each release and contact details for them are provided, so should someone have concerns over the statistics they are able to communicate with us. Methodology documents are published to enable users to provide scrutiny.

As noted in Section 2, the accuracy of the data collected is checked with the member of personnel before being inputted to the database. Population Statistics Division (PSD) do not rely on any further checks being carried out on the data before it is supplied. The checks carried out by PSD upon receipt of the data are detailed in Section 5.

This section details the checks and standards applied to the data after receipt by PSD. The checks carried out prior to receipt of the data are detailed in Section 4.

Checks are carried out on all components of population change (including the Foreign Armed Forces component covered in this report) to ensure that the data are accurate enough to be included in the mid-year estimates. Standard checks on this component include:

• the data relates to service personnel
• the file contains all required variables in the correct format
• the sex or age distribution is similar to that of previous years
• the Local Authority distribution of the data is similar to that of previous years

Should the data have changed significantly, then several years will be investigated to see if it is reflective of a longer trend. If this does not prove to be the case then USAF would be contacted for confirmation and/or investigation.

6.1 Strengths and limitations

Overall, this data source is judged to be of good quality for the use to which it is being put in the population estimates:

• coverage is comprehensive, with a requirement for all service personnel to be included
• it is expected to be up-to-date with no material issues of lags between someone joining or leaving a base and this being reflected in the data

However, there are some acknowledged limitations in the data:

• the data has been collected for administrative purposes and no specific checks are made that each member of personnel meets the target concept of usual resident
• the information relates to base rather than place of residence, and the latter must be derived using a base-residence matrix from the 2011 Census

6.2 Justification of rating

Population estimates are used to make decisions on resource allocation by central and local government and as such are assessed as having a relatively high public interest profile. However, the US Armed Forces are a small part of the population and any inaccuracies in the data would have a similarly small impact on the quality of the population estimates.

Though a formal agreement for provision of data is not in place, its supply is part of an established and standard procedure and the nature of the data collected is such that it is judged unlikely that there would be a material change in definitions or information collected which would impact on the quality of the estimates. We have therefore judged that the appropriate level of assurance for this data source is A1 – Basic Assurance.