Quality assurance of administrative data report for manufacture of basic iron and steel

1.1 Background

National Accounts and Economic Statistics (NAES) within the Office for National Statistics (ONS) collect data from the International Steel Statistics Bureau (ISSB) on activities for the manufacture of basic iron and steel. These data form one source in the calculation of short-term economic output indicators, namely gross domestic product (GDP (O)) and Index of Production (IoP) for the UK.

This report outlines the process data take from initial collection through to the output of the release. It identifies potential risks in data quality and accuracy as well as details of how those risks are mitigated.

This report forms the latest in a series of quality assurance of administrative data (QAAD) reports produced by NAES to investigate the administrative data sources we use in the production of short-term economic output indicators as set out by the UK Statistics Authority. This report specifically focuses on our administrative data use for the basic iron and steel industry (SIC 24.1-3) and does not cover SIC 24.4-5 – other basic metals casting.

Separate industries where we apply administrative data will be considered in other QAAD reports in the series.

Further information relating to quality and methodology for the short-term economic output indicators can be found in our Gross domestic product, preliminary estimate report and our UK Index of Production Quality and Methodology Information report.

1.2 Standard Industrial Classification (SIC) Overview

The activities of manufacture of basic metals cover all activities under UK SIC 2007 division 24.

Based on the “UK Standard Industrial Classification (2007)” the industry is classified to five groups:

24.1 – Manufacture of basic iron and steel and of ferro-alloys

24.2 – Manufacture of tubes, pipes, hollow profiles and related fittings, of steel

24.3 – Manufacture of other products of first processing of steel

24.4 – Manufacture of basic precious and other non-ferrous metals

24.5 – Casting of metals

We are interested in iron and steel, excluding other metals, and investigating the administrative data source feeding in SIC 24.1, 24.2 and 24.3.

According to the Inter Department Business Register (IDBR)¹ there were 1,190 enterprises classified under SICs 24.1, 24.2 and 24.3 in September 2017. This is an increase of approximately 95 enterprises (8.7%) from the previous year.

Most of the enterprises within these three SICs for 2017 were allocated to 24.1, with 800 enterprises (67%). Within the three SICs, there were 790 enterprises with fewer than five employees (66%), compared with 185 enterprises with 20 or more employees (15%).

Of the 1,190 enterprises in the three SICs, the annual turnover of 745 (62%) enterprises were below £200,000.

980 (82%) enterprises had registered turnover below £2 million, with only 20 enterprises in total having registered turnover over £50 million.

The figures above are based on a download of IDBR and rounded to the nearest five. These data do not align with information provided by ISSB. ISSB has many fewer businesses classified to SICs 24.1 to 24.3 than ONS has on the IDBR. It is possible that IDBR contains misclasses. ISSB has more accurate data, however, even ISSB may be missing some businesses, so may not be 100% trustworthy. This requires further work. ISSB may be missing some businesses but is close to 100% trustworthy. This makes the ISSB data source extremely valuable to the ONS as a much more accurate data source than our own.

Notes for: Introduction

1. The Inter-Departmental Business Register (IDBR) is a comprehensive list of UK businesses that is used by government for statistical purposes. It provides the main sampling frame for business surveys carried out by both the ONS and other government departments. It is also an important data source for analyses of business activity.

2.1 UK Statistics Authority QAAD toolkit

The assessment of our administrative data sources has been carried out in accordance with the UK Statistics Authority Quality Assurance of Administrative Data toolkit.

Each administrative data source investigated has been evaluated according to the toolkits risk and profile matrix (Table 1) reflecting the level of risk to data quality and the public interest profile of the statistics.

The toolkit outlines four specific areas for assurance and the rest of this report will focus on these areas in turn. These are:

• operational context and administrative data collection

• communication with data supply partners

• quality assurance principles, standards and checks applied by data suppliers

• producer’s quality assurance investigations and documentation

In the assurance of our data source, we have chosen to give a separate risk and profile matrix score (Table 1) for each of the four areas of assurance. This will allow us to focus our investigatory efforts on areas of particular risk or interest to our users (Table 2).

2.2 Assessment and justification against the QAAD risk and profile matrix

The risk of quality concern and public interest profile has been set as “low” due to the small contribution that the iron and steel statistics feed into the Index of Production (0.64%) and gross domestic product (0.1%).

As such, a score of A1 is deemed appropriate for this data source.

All scoring was based on the level of risk of the data and interest of our users. Results for each area of assurance for iron and steel are shown in Table 2. If you feel this report does not adequately provide this level of assurance or you have other feedback, please contact stoi.development@ons.gov.uk with your concerns.

3.1 Operational context and administrative data collection (QAAD matrix score A1)

This relates to the need for statistical producers to gain an understanding of the environment and processes in which the administrative data are being complied and the factors which might increase the risks to the quality of the administrative data.

National Accounts and Economic Statistics (NAES) obtain monthly data from a survey the International Steel Statistics Bureau (ISSB) administrate. The ISSB is a company limited by guarantee. ISSB are part-owned by UK Steel and part-owned by Tata. Members of these two entities send data to them. ISSB are a leading supplier of global trade data for steel and raw materials. They output a series of articles and analysis to their website throughout the year and produce statistical and economic reports on the data they receive. They maintain a detailed database of the imports and exports of steel and steelmaking raw materials for more than 50 steel producing nations, collectively accounting for 97% of global steel output. This high level of coverage also allows an accurate assessment of the trade flows for those countries for which we cannot obtain national trade data. 100% of the companies that make iron and steel in the UK are included, however, there are some companies making downstream products that they do not collect data for.

Their database consists of monthly import and export data for over 50 steel producing nations split by 6-digit Harmonised System tariff code. They have volumes (in tonnes), and values (cost insurance and freight prices for imports and free on board prices for exports).

This monthly data is important to NAES as a guide for short term indicators.

Most of the data is collected online via their central computer system (see Note). Some steelworks fill in forms on Microsoft Excel which are sent via email.

Businesses have guidance notes on how to complete the forms. They were, however, written in the late 1980s when the method of sending data to ISSB, via forms, was created. They explain product definitions so that the correct tonnage or value is reported each month. ISSB assumes each company still apply the same rules, with knowledge of these procedures being passed down internally throughout the years.

ISSB state that the overall industry reports they send to government are as representative as possible, as energy data is a legal requirement. There is, however, no legal or contractual obligation for individual businesses to send ISSB data. Unless the business receives monthly reports from ISSB – which some do – then it is in their best interest to send ISSB data for ISSB to collate this information for reporting purposes. Tata, Liberty Steel and British Steel are the largest companies ISSB work with. Pressure can also be applied by UK steel. ISSB chases non-responders who have not returned figures by the second week of the month.

Note:

This is a central computer system used to collate information from businesses within a company registered for VAT in the UK which must declare despatches and arrivals within the European Community. It processes financial and statistical data on behalf of the Single European Market but also:

• sales and purchases of steel industry products, raw materials, fuel and energy items; this is for all transactions within the UK and European Community and countries in the rest of the world

• summarised financial data for sales and purchases of goods and services to be declared to HM Customs and Excise on a UK Value Added Tax (VAT) return

• summarised financial data for all transactions, external or internal

Tata and British Steel use this system, which allows ISSB to obtain delivery information from them. Liberty Speciality Steels no longer use this system due to changes in their computer software, and instead use “forms” via Excel which they need to fill in manually each month.

Strengths:

• 100% coverage
• non-responders chased
• data is a legal requirement sent to government

Weaknesses:

• guidance notes from 1980s
• some data collected via Excel spreadsheet
• no legal or contractual obligation for individual businesses to send ISSB data

3.2 Communication with data supply partners (QAAD matrix score A1)

This relates to the need to maintain effective relationships with suppliers (through written agreements such as service level agreements or memoranda of understanding), which include change management processes and the consideration of statistical needs when changes are being made to relevant administrative systems.

The International Steel Statistics Bureau (ISSB) contact businesses directly if they wish to get figures corrected or explained. They chase non-responders by phone and e-mail if businesses have not returned figures by the second week of the month.

Certain reports have a set deadline so ISSB ensure that all the data is received on time, to provide an efficient service.

Both their CEO and Contracts Manager attend presentations throughout the year. The Contracts Manager distributes regular newsletters to customers and prospective customers via email. Their website is kept up to date with the latest trade information which customers can access via the online trade enquiry system, which they subscribe to.

ISSB have a contract with the Office for National Statistics (ONS) and a service level agreement in place with National Accounts and Economic Statistics (NAES).

NAES receives two emails a month from ISSB. The first contains preliminary data and the second the final report. NAES receives these two data deliveries and did meet face-to-face in early 2018, however, it has no regular contact with ISSB. The only time NAES contact ISSB is to chase late returns or query anomalous data. There has been a couple of ad-hoc requests outside of this, however these are in no way regular.

Strengths

• service level agreement with ONS
• service contract with the Department for Business, Energy and Industrial Strategy (BEIS)
• regular monthly data and report
• website kept up to date with latest trade information

Weaknesses

• no regular communication with NAES

Next steps

• NAES to maintain newly established communication links developed during the writing of this report

3.3 Quality assurance principles, standards and checks by data supplier (QAAD matrix score A1)

This relates to the validation checks and procedures undertaken by the data supplier, any process of audit of the operational system and any steps taken to determine the accuracy of the administrative data.

International Steel Statistics Bureau (ISSB) carry out the following quality assurance checks:

• returned data are automatically processed through an Excel macro, which transfers the data to a text file which is uploaded to their database

• ISSB have a specially designed programme through Microsoft Access called Client on which all the data is stored

• reports are built and ran through Business Objects (specialist software for producing reports), which is linked to Client

ISSB produce automated reports that alert them to any figures that appear higher or lower than expected. Some of these reports are run through Business Objects, to compare certain form numbers against each other to ensure a match. Client also runs its own checks; when the data is being uploaded a validation report is automatically produced, which highlights if the sum of values is correct, if data should or should not be in a certain place and so on.

This report also compares the current month’s figures to previous months to ensure that there are no anomalous figures – if there are they check the figures that have been uploaded against what they have been sent. They would then contact the individual who sent the data asking them to double-check that it is correct. They often send NAES revisions, corrections or an explanation for why a certain product is higher or lower that month, for example, summer holiday, maintenance work on a furnace, and so on.

Changes are always discussed amongst the ISSB team with manager approval. There have been some changes in the last couple of years due to Tata selling a lot of their UK assets to British Steel and Liberty Steel. Tata are also changing the way in which they send their data to ISSB. They have regular discussions with those involved at Tata either in person or over the phone.

In addition to checking figures and contacting the data provider to query anomalous figures, ISSB also conduct an annual audit to ensure all data looks correct with monthly figures and previous annual data.

ISSB state: “There can be slight discrepancies due to product definitions but by and large, the data is an accurate representation of the market.”

There are some monthly comparisons and revisions due to late responding steel-making businesses, but they are very small.

Strengths

• automated checks
• error reports produced
• monthly comparisons and revisions
• two-way communications

Weaknesses

• possible slight discrepancies due to product definitions

3.4 Producers quality assurance investigations and documentation (QAAD matrix score A1)

This relates to the quality assurance conducted by the statistical producer, including corroboration against other data sources.

National Accounts and Economic Statistics (NAES) use trade data to produce aggregate figures for SIC 24.1 to 24.3. NAES pick up the previous year’s figure to check for revisions; they state the figures are usually revised, but the revisions are very small.

NAES check the data for anomalies and when they find something amiss they investigate and contact the provider as required. The data are then entered on their spreadsheet, which weights and produces an index. The data is re-checked. If NAES are satisfied, all figures are uploaded to CORD (an internal ONS system) for the final output. NAES run with the data and compare with their forecast.

There are clear desk instructions for each step of the above processes.

NAES has limited communication with the International Steel Statistics Bureau (ISSB) and no communication with individual iron and steel making business or organisations, so there is little or no user engagement and limited feedback on the use of iron and steel statistics has been provided.

Strengths

• detailed instructions for producing figures, weights and index

Weaknesses

• limited quality assurance by NAES

• communication block

In investigating the administrative source for the activities of iron and steel, National Accounts and Economic Statistics (NAES) consider the main strengths of the data for our purpose to be:

• extensive quality assurance by International Steel Statistics Bureau (ISSB)

• full 100% census, with 100% response

• detailed knowledge and expertise by ISSB

• detailed Office for National Statistics (ONS) instructions for producing figures

• existing service level agreement in place between ONS and ISSB

We believe current limitations of this data source are:

• no real communication between ONS and ISSB; this should be addressed by establishing links with the ISSB and initiating regular contact

• use of excel

• no legal obligation for suppliers to submit data

In constantly seeking to improve our data sources we will be taking steps to address these limitations and these will be communicated to users in future quality assurance of administrative data (QAAD) report updates for this topic.

Despite these limitations, based on the low risk of quality concerns and small contribution that the manufacture of basic iron and steel statistics feed into Index of Production (0.64%) and gross domestic product (0.1%), NAES consider this data source to fulfil the requirements of an A1 assurance rating.