Data archiving policy

The Data Archiving policy sets out clear principles for the management of statistical data to data archives within the Office for National Statistics (ONS).

Data can be archived internally and managed by the ONS, or archived externally, such as at The National Archives (TNA).

This policy applies to all Office for National Statistics (ONS) and UK Statistics Authority employees, including staff on fixed-term, temporary or permanent contracts, staff on secondment, students, and contractors.

This policy applies to statistical data generated by the Office for National Statistics (ONS) as part of the statistical production process. It also applies to data that has been sourced or acquired for production and/or research purposes.

Data archiving ensures ongoing availability of data for future research purposes. Data will be archived internally at the ONS, and assessed for suitability for permanent preservation at The National Archives (TNA) 20 years following the creation date of the most recent dataset.

Personal data is retained in accordance with the UK General Data Protection Regulation (UK GDPR) and the Data Protection Act (DPA) 2018.

The Office for National Statistics' (ONS) data can be held online or offline. Usually, data will be available online in live systems for production and/or research purposes. When data is no longer required to be held online (for example, because the speed of access to those datasets is not a critical factor), they can be held in offline data archives.

This policy is to ensure that data archiving is considered, whether that is internally within the ONS or to external archives, in order to proactively preserve data assets for the long-term. This removes the need for frequent database backups and improves processing speeds on live systems.

Retention periods must be recorded in the ONS Information and Data Asset Register to ensure that regular retention reviews for all data recorded on live systems and internal ONS archives are carried out as part of data governance activities.

Data archived internally within the ONS must be supported by a Digital Preservation and Digital Continuity process.

This policy supports the Data Principles which underpin our Data Strategy.

This policy applies to all data that the Office for National Statistics (ONS) holds to undertake its statutory functions as a data controller. This includes statistical data and any supporting metadata, for example data obtained through mandatory and compulsory surveys, or data obtained from administrative sources in the public or private sectors.

Data archives need to be managed in a consistent way across the ONS. For example, the data should be managed and protected while held in data archives, and associated metadata should be maintained to support the management of data archives and context. Digital preservation and digital continuity measures should be adopted to ensure sustained access and usability over time.

Data and metadata should be archived to the ONS internal archive when they no longer need to be held online but still need to be held for future research purposes, or because of a lengthy retention period. Retention of statistical data will be managed according to the Statistical Data Retention and Disposal Policy. Managing archives centrally will leverage reuse and reduce data duplication.

Datasets retrieved from the internal ONS archive will be accompanied by a manifest file. This is to ensure the metadata is supplied for the datasets being retrieved, in compliance with our manifest file data standards.

Archived datasets that are no longer required, that are not worthy of permanent preservation at The National Archives (TNA), or which cannot be held long-term for legislative purposes will be disposed of securely in accordance with ONS policy and best practice. Disposal of statistical data will be managed according to the Statistical Data Retention and Disposal Policy.

Access to archived data needs to be authorised and approved (taking into consideration factors such as the regulatory and legislative constraints, the user role, and the sensitivity of the archived data).

Access should be provided in a timely fashion, in accordance with agreed service level agreements. Access should be managed at a level of granularity which allows administrators to prevent unauthorised access to archived data.

Data quality is governed across the entire lifecycle, up to and including data archiving and disposal, to ensure it is appropriately managed. Quality of archived data is managed according to our Data Quality Management Policy.

The Information Asset Owner (IAO) and Information Asset Liaison Officer (IALO) role holders are accountable and responsible for data governance activities assigned to them as part of their appointment to the role. Data governance duties relating to this policy include:

• managing data assets throughout the data lifecycle, and ensuring that related governance metadata is recorded, accurate, and complete

• undertaking reviews of metadata when prompted, and decision-making for ongoing access to the data when retention periods are reached

• ensuring a data sensitivity assessment has been undertaken for assigned data assets, and that associated risks are managed accordingly

• responsibility for decisions on use, transfer, and access requests for data assets; oversight of associated processing activities

• responsibility or accountability for risks associated with the data assets

• decision-making in relation to project or user accreditation relating to access to data

The Knowledge and Information Management (KIM) team is accountable to the Chief Security Officer and responsible for:

• approving user access to data held in the data archives after consulting with the Information Asset Owner (IAO)

• ensuring that the governance metadata relating to data assets is accurate on the asset register and associated catalogues

• supporting business areas with lifecycle management of information and records

• ownership of the interface with The National Archives (TNA) for the permanent preservation of selected public records, which includes datasets