Quality assurance of administrative data report for activities of trade unions

1.1 Background

The National Accounts and Economic Statistics (NAES) Group within the Office for National Statistics (ONS) download data from the Department of Business, Energy and Industrial Strategy (BEIS) on activities of trade unions membership. These data form one source in the calculation of short-term economic output indicators, namely gross domestic product (GDP(O)) and index of services (IoS) for the UK.

This report outlines the process data take from initial collection through to the output of the release. It identifies potential risks in data quality and accuracy as well as details of how those risks are mitigated.

This report forms the latest in a series of quality assurance of administrative data (QAAD) reports produced by NAES to investigate the administrative data sources we use in the production of short-term economic output indicators, as set out by the UK Statistics Authority Quality Assurance of Administrative Data. As such, this report focuses on our administrative data use for the activities of trade unions industry, specifically SIC_94.2, and does not aim to cover other aspects of SIC_94, as these are collected using non-administrative data sources. Separate industries where we use administrative data will be considered in other QAAD reports in the series.

Further information relating to quality and methodology for the short-term economic output indicators can be found in our Gross domestic product, preliminary estimate: Oct to Dec 2016 report and our Index of Services Quality and Methodology Information.

1.2 Standard industrial classification (SIC) overview

The activities of membership organisations cover all activities under UK SIC 2007 division 94. This includes activities of organisations representing the interests of special groups or promoting ideas to the general public. These organisations usually have a constituency of members, but their activities may involve and benefit non-members.

Based on the ‘UK Standard Industrial Classification (2007)’ the industry is classified to three groups:

• 94.1 – Activities of business, employers and professional membership organisations
• 94.2 – Activities of trade unions
• 94.9 – Activities of other membership organisations

According to the Inter-Departmental Business Register (IDBR)¹ there were 21,345 enterprises classified under division 94 in March 2016. This is an increase of approximately 555 enterprises (2.7%) from the previous year (March 2015).

The majority of enterprises within division 94 were allocated to “94.9 – other membership”; which equates to 17,565 enterprises (82.3% of the total division).

Within division 94, there were 14,515 enterprises with fewer than 5 employees (68.0% of the division), compared with just 1,310 enterprises with 20 or more employees (6.1% of the division).

Of the 21,345 enterprises in division 94, the annual turnover of 15,690 (73.5%) enterprises was below £250,000. For the vast majority of division 94 enterprises within, 20,665 (97.7%) had registered turnover below £2 million, with only 30 enterprises in total having registered turnover of over £50 million.

There were 325 enterprises allocated to 94.2 in March 2016. This is a decrease of 10 enterprises from the previous year (March 2015). Group 94.2 has the smallest number of enterprises within division 94 (1.5% of the total division).

Notes for: Introduction

1. The Inter-Departmental Business Register (IDBR) is a comprehensive list of UK businesses that is used by government for statistical purposes. It provides the main sampling frame for business surveys carried out by both the ONS and other government departments. It is also a key data source for analyses of business activity.

2.1 UK Statistics Authority QAAD toolkit

The assessment of our administrative data sources has been carried out in accordance with the UK Statistics Authority Administrative Data Quality Assurance toolkit.

Each administrative data source investigated has been evaluated according to the toolkits risk and profile matrix (Figure 1) reflecting the level of risk to data quality and the public interest profile of the statistics.

The toolkit outlines four specific areas for assurance and the rest of this report will focus on these areas in turn. These are:

• operational context and administrative data collection
• communication with data supply partners
• quality assurance principles, standards and checks applied by data suppliers
• producer’s quality assurance investigations and documentation

In the assurance of our data source, we have chosen to give a separate risk and profile matrix score (Table 1) for each of the four areas of assurance. This will allow us to focus our investigatory efforts on areas of particular risk or interest to our users (Table 2).

2.2 Assessment and justification against the QAAD risk and profile matrix

The risk of quality concern and public interest profile has been set as “low” due to the small contribution that the trade union statistics feed into index of services (0.02%) and gross domestic product (0.01%). As such, a score of A1 (basic assurance) is deemed appropriate for this data source.

All scoring was carried out by National Accounts and Economic Statistics (NAES) Group based on the level of risk of the data and interest of our users. Results for each area of assurance for trade union membership are shown in Table 2 .If you feel that this report does not adequately provide this level of assurance or you have any other feedback, please contact stoi.development@ons.gov.uk with your concerns.

3.1 Operational context and administrative data collection (QAAD matrix score- A1)

This relates to the need for statistical producers to gain an understanding of the environment and processes in which the administrative data are being complied and the factors which might increase the risks to the quality of the administrative data.

Trade union data published by the Department for Business, Energy, Industrial Strategy (BEIS) is used primarily by index of services (IoS) and gross domestic product (GDP) teams in ONS. The data are specifically used to measure trade union membership numbers within the UK.

BEIS are responsible for publishing National Statistics on trade union membership and the statistics are published annually. Official government statistics on trade union membership have been collected on a regular basis since 1892 from Department Employment Statistics. From 1974, the statistics are collected by the Certification Officer (CO) by law. The data BEIS uses for their publication is uplifted directly from the Annual Report of the Certification Officer, which is published on the government website.

The statistics on trade unions, which the Certification Officer (CO) publishes in their annual report, are taken from the annual financial returns known as an AR21 form, which all active trade unions in Great Britain are obliged under the Trade Union and Labour Relations (Consolidation) Act 1992 to complete and send to CO office.

The collection process for trade unions is to complete the AR21 form for annual return for trade unions , which can be downloaded from the GOV.UK website. They then follow guidance notes and complete a checklist in the completion of the return.

In England and Wales the forms are directly returned in paper format to the CO. In Scotland, forms are returned to the CO via the Assistant Certification Officer (ACO) for Scotland, who is responsible for scanning all the returns and emailing all copies to the CO office.

The CO have a robust system to ensure coverage, which includes updated reports to show unions that have not submitted a return by the due date and ensures no returns are overlooked from English, Welsh or Scottish trade unions. Wide coverage by the CO is ensured as any union that does not submit an annual return is liable to prosecution. The strengths of this method are:

• data has been collected by law since 1892 and since 1974 by the CO, providing a time tested and full coverage data source
• standardised forms, guidance notes and checklists are provided by the CO for trade unions to complete, improving accuracy of submission
• CO has robust checks for non-respondents in place and ability to prosecute if necessary, maximising coverage

The NAES aim to maintain this level of knowledge of this area.

3.2 Communication with data supply partners (QAAD matrix score A1)

This relates to the need to maintain effective relationships with suppliers (through written agreements such as service level agreements or memoranda of understanding), which include change management processes and the consideration of statistical needs when changes are being made to relevant administrative systems.

The Certification Officer (CO) confirmed they meet unions individually, at their request, to discuss the implications of recent legislative changes and from time to time write to unions en masse to inform them about new reporting requirements. They also supply telephone contact details on the AR21 forms to answer any difficulties or problems in the completion of their returns, in addition to the guidance notes and checklists provided.

The CO and Business, Energy, Industrial Strategy (BEIS) publish annual reports with information about the data collection and methodology for users, such as the Annual Report of the Certification Officer 2015 to 2016 and the Trade Union membership Statistical Bulletin 2016. BEIS also communicate with their users by requesting feedback on its trade union statistics within their releases.

The CO, BEIS and National Accounts and Economic Statistics (NAES) Group argued there is no requirement for regular direct contact between departments and no formal data agreements are in place, as there is a lack of any legal obligation to share the trade union data with NAES. Instead, all the data is shared voluntarily as result of goodwill. The logistics of implementing and actively managing such formal arrangements are considered both prohibitive and unnecessary for this data source, as voluntary data provision has consistently resulted in an annual data sample since 1974.

The strengths of this method are that it provides contact details and guidance on AR21 forms; CO meets with unions to discuss concerns and writes en masse to report new requirements; and BIS publish annual report with technical notes for users. The weaknesses, however, are that there are no formal regular meetings between NAES, BEIS and CO, and it is missing possible methodology changes or law changes that could affect data. The next step is to establish more regular contact with BEIS and CO contacts.

3.3 Quality assurance principles, standards and checks by data supplier (QAAD matrix score A1)

This relates to the validation checks and procedures undertaken by the data supplier, any process of audit of the operational system and any steps taken to determine the accuracy of the administrative data.

In quality assurance of the data it is a legal requirement for all trade unions, except for a small number of very small unions, to have their accounts independently audited by an auditor qualified under the Companies Acts. It is also a legal requirement for the auditor to sign the union’s annual return to confirm that it provides “a true and fair view” of the matters to which it relates. In respect of the membership information for trade unions it is also a legal requirement for unions with more than 10,000 members to appoint an ‘assurer’, who must be an independent person qualified under the provisions of the Transparency of Lobbying, Non-Party Campaigning and Trade Union Administration Act 2014. The assurer must provide a certificate to the Certification Officer (CO) each year, certifying that the union has complied with its statutory duty to maintain a register of its members and to keep that register accurate and up to date so far as reasonably practicable. Unions with less than 10,000 members must appoint an officer of the union who is authorised to fulfil the same role as the assurer for larger unions, and it is their responsibility to provide the CO with an equivalent certificate. More information can be found in the Guidance on trade union register of members and membership audit certificate requirements.

Staff within the CO’s office extract data from each annual return and manually input these figures into an internal database. There are no built-in automated checks but the CO runs scrutiny checks on all figures in the annual returns. The CO uses the database to produce the tables and information contained in the CO’s annual report. The CO does not verify individual’s details within the annual returns but does carry out sense checks to ensure each return contains all information required by statute and that all values add up. Where differences are identified, the CO raises any queries with unions and its auditors to discuss and resolve concerns before publishing any figures, following which the CO may revise the data. The CO told us that this is very rare and in such cases usual practice is to issue an errata notice in the following year report for users. Business, Energy, Industrial Strategy (BEIS) confirmed any revised figures would be picked up for their annual report prior to publication.

BEIS confirmed there are no further validations or quality assurance steps in place to check the accuracy of the trade union data due to the independent checks already carried out by qualified assurers. The data are directly uplifted from the CO’s annual report and published on the BEIS website.

BEIS are responsible for publishing the National Statistics on trade union membership and has considered using extracts of the CO data within their databases where they could perform further checks; however, they feel this would be disproportionate leading to a less timely dataset and may affect quality and consistency of the longer time series.

The strengths of this method are that trade union data is independently audited by qualified auditors, assurers and officers of the unions and trade union annual returns are available to view on the website. However, the CO’s database has no automated upload and checking procedure.

3.4 Producers quality assurance investigations and documentation (QAAD matrix score A1)

This relates to the quality assurance conducted by the statistical producer, including corroboration against other data sources.

User engagement is continual and the feedback tends to relate to the overall impact of the statistics rather than to the individual data source used, and to date no specific feedback on the use of trade union statistics has been provided.

National Accounts and Economic Statistics (NAES) Group downloads the data from the Business, Energy, Industrial Strategy (BEIS) website around April each year using detailed desk instructions. NAES confirmed for their purposes this is considered as a long-time lag for data feeding into short-term indicators. The data is loaded into systems with no further checks applied. A simple calculation is then run to aggregate the figure into the higher level SIC94, and the figures for SIC94, 95 and 96 are then combined for users. The process to follow is fully described by desk notes. In the event of missing data imputation is built into the system that forecasts a figure based on previous years data. NAES confirmed the trade union figures including revisions are compared with those for previous years.

NAES deem the trade union data source to have a very low proportional impact on the statistics that feed into index of services (0.2%) and gross domestic product (0.1%) and feel this level of assurance is carried out throughout the process but particularly earlier by the independent auditor is sufficient and proportionate to its importance.

The strengths of this method are that it provides desk notes for users and figures including revisions are compared with those for previous years. However, no further quality assurance checks are applied, there is a 12-month time lag, and no log has been set up for an audit trail of data used in previous years. The next steps for NAES are to consider new data sources to eliminate time lag.

In investigating the administrative source for the activities of trade unions, National Accounts and Economic Statistics (NAES) Group consider the main strengths of the data for our purpose to be:

• detailed knowledge of the subject area by the Certification Officer (CO)
• a dataset that is collected by law, providing full coverage
• standardised forms, guidance notes, checklists and contact details by the CO improving accuracy of submission
• trade union data independently audited by qualified auditors, assurers and officers of the unions
• NAES comparisons of data to those from previous years and full desk instructions

We believe that current limitations of this data source are:

• a lack of an audit log by NAES of data used in previous years
• no direct, formal, regular meetings between NAES and CO
• time lag as data is published annually

In constantly seeking to improve our data sources, we will be taking the next steps to address these limitations and these will be communicated to users in the future quality assurance of administrative data (QAAD) report updates for this topic.

However, despite these slight limitations, based on the low risk of quality concerns and small contribution that the trade union statistics feed into index of services (0.02%) and gross domestic product (0.01%), NAES consider this data source to fulfil the requirements of an A1 assurance rating.