Our Data Strategy

In line with the radical and ambitious principles of the Office for National Statistics (ONS) strategy, the ONS is moving to a new phase as both a data and technology leader, alongside the analytical powerhouse that it already is.

There is a trend of falling response rates to surveys, and people are more in control of their data and how they wish to share it.

This strategy sets out a vision for the ONS to pivot towards realising the full value of data as a strategic asset, while maintaining high levels of trust and transparency.

This strategy is deliberately high-level and simple. It explains what data capabilities and foundations we will focus on through eight missions to be the most trusted, joined-up and data-driven organisation in the public sector.

Each mission has a series of commitments which will be implemented over the next three years and against which key performance indicators (KPIs) will track progress. We are already widely recognised as the "original" data organisation, and it is our data expertise and reputation which sets us apart.

The ONS is highly trusted by the public, as well as by our private and public sector colleagues. We are respected for our data handling experience and revered by our public sector partners. There is no other government department that has the trust or the experience in data that the ONS has, nor which handles the scale of data that the ONS does.

This strategy sets out how we will keep it that way, while responding to a more complex, digital, and inclusive society.

Fiona James
Chief Data Officer and Director, Data Growth and Operations

The last Office for National Statistics (ONS) data strategy set out a series of technical principles and standards that we expect adherence to when working with data. This strategy takes a broader perspective, covering the full range of data capabilities and foundations the ONS is prioritising to strengthen its data and technology leadership.

In particular, increasing public trust in how we gather, manage, and use data, from an already high level, underpins this data strategy, and is at the core of all of our work with data.

There are three main areas in which we've seen a step change around data since the last strategy, caused by the coronavirus (COVID-19) pandemic as an accelerator.

Data is an essential input

We are handling more data than ever before to meet the ONS and government needs, and we have an ambition to go further, treating data as an essential input to the way that we operate.

The Data Access Platform, the ONS's internal data platform, now holds over 2,000 datasets and the Secure Research Service, which facilitates data access for external researchers, over 120. We have seen a 50% growth in data assets over the last two years, with a greater variety of data, which is both real-time and relevant, and enables our key outputs around the population, the economy, health, crime, business and society. We are at the heart of mobilising and informing data for the main decision-makers on every important topic; for example, you will have seen our recent consultation on the future of population and migrations statistics, which puts administrative data at the heart of a transformed system.

This strategy calls for the ONS to treat data as an essential input and central to how we operate in the future. For example, to what extent can we use technology

to interoperate with businesses in the future, such that we reduce the need for business surveys?

Not only does the ONS need to go further to make the end-to-end data journey across the organisation as smooth as possible, but it also needs to place more emphasis on the importance of data models to support integrity when using future automation and artificial intelligence. Data models will help the ONS to understand and join-up data better to generate greater efficiency, recognising that our complete data portfolio is much greater than the sum of its parts. Instead of seeking new data for new problems, the ONS can re-use existing data and find new and quicker ways of engaging with the public and respondents.

The three Ls of data

The three Ls of data - local, longitudinal and linked - will be crucial features of the use of data going forward. These Ls are defined as:

• local- its importance for geospatial analysis, aggregation, and at the individual level

• longitudinal- the value we gain from longitudinal datasets, such as education or in relation to equality

• linked- an increasing scope and scale of linkage and a collectively enriched set of integrated data assets around important themes like climate change, health, economy, and science

Combining a growing set of linked data assets with longitudinal analysis, which can be analysed at a local level, has exponential power far greater than we have today. All of our functions, including security, data ethics, and quality, will respond to this step change.

Modern, secure tools and technologies

We have more modern, secure tools and technologies which enable us to transform the way we access and integrate data. Our ambition is to use the cloud to enable federated data models, and to use the Reference Data Management Framework (RDMF) to handle data securely, but more consistently for linkage purposes, where it is appropriate to do so.

Data linkage is an evolving area - one in which we are experts - and where we are already leading the way. The RDMF is an ambitious framework made up of component parts to index data and match it quickly and consistently, at scale, with an understanding of the level of quality. It is an ONS product, available for government use, and has the potential to transform data integration, bringing efficiencies through innovation.

The RDMF is already an important pillar of how we understand businesses in the UK and how we intend to transform future population and migration statistics. It is the ambition of this strategy that the ONS implements this product across government in order to ensure that sensitive data is kept secure and is used ethically.

The Integrated Data Service (IDS) is the next generation research environment. It will provide the most secure, cutting-edge technology available to analyse and integrate data at scale.

Using our collective analytical power, including through the data science campus, we will design and shape this next generation research environment to future proof and respond to analytical needs.

We imagine that we are at the "Rosetta Stone" moment of our time, where we no longer talk of data sharing but instead of accessing data at source, where we can communicate across the different languages of cloud across departments.

For example, we can interoperate across Azure and Google infrastructures, and analysts can bring this together "on their tablets" to answer questions we do not yet know we need to ask.

Having metadata and efficient, automated data pipelines and operations will be the primary foundations for improving data usability and data quality. We will lead work in this area not just to join up ONS metadata, but also to make ONS data the highest quality, in line with our standards and expectations.

It is critical to our success that we maintain trust and transparency, and that we understand the different values, perspectives, and cultures when working with data, and engage in new and meaningful ways which communicate our new approaches. We recognise the important role that the ONS has to play to demystify the technical data jargon and ensure that everyone understands how we keep data safe and secure.

Developing data skills

The ability to grow data dexterity to handle and manage more complex data is now an essential part of every one of the 5,000 ONS employee roles.

To become a truly data-driven system, we need a breadth of skills across the ONS that are transferable across both the ONS and government. Data professional roles are now established within the digital, data, and technology framework (DDaT), and in the ONS we are committed to providing learning and development, continual improvement, and innovation to develop and sustain technical and non-technical careers. We will not achieve the ambition in this strategy without continuing to improve the level of data skills across the ONS.

Data integration and architecture

Led by Office for National Statistics (ONS) Chief Data Architect

Our mission

To enhance data in the ONS and across government by making it more findable, accessible, interoperable and re-usable (FAIR), as well as better linked.

Our commitments

We will effectively re-use datafrom across government and beyond to underpin our statistical and analytical work, delivering more efficiently across the organisation and providing value for money.

We will ensure that metadata is at the heart of our work to help make our data findable and re-usable and allowing visibility of the limitations of the data. To support this, we will implement a central strategic metadata solution to allow discovery and re-use of data and facilitate streamlining of the data journey.

We will design, architect and deliver data assets through the Integrated Data Service (IDS) in an integrated, easily re-usable form, including the provision of Integrated Data Assets and Products where appropriate.

We will implement our Reference Data Management Framework (RDMF),which brings together our core indexes (Business Index, Demographic Index, and Address Index) by rolling out the matching services to the ONS's statistical data. This will reduce the amount of data linkage performed on a case-by-case basis, by indexing individual datasets to allow for easy and flexible linking at the point of use. To support this, we will automate indexing of data insofar as is possible and do this as early in the data journey as possible. We will agree to the cross-government ambition for a cross-government RDMF matching service.

We will lead the establishment and implementation of data standards for the ONS and across government, with more robust governance in place.

Definitions of success

Designed an ONS data model which captures the analytical value of all the data that the ONS needs so that we understand the value of data as an input. To support this, we have implemented a strategic metadata solution which will help data discovery and reuse. Quality meta-data is maintained at source and widely available.

Designed and delivered critical national integrated data assets for the most important public policy areas, for example Net Zero, Levelling Up and Health, aiming for more as our new systems mature, which are made available via IDS.

IDS is used as the preferred platform for research on cross-government data, with many processes automated to deliver integrated, interoperable data products.

The RDMF is providing high-quality reference data and indexing services about businesses, addresses, demographics, classifications, and geographies.

Robust governance of data and metadata standards within the ONS and as part of cross-government communities and fora, leading to increased data interoperability across the government data landscape.

Where we will be in three years time

The ONS will have integrated a wide range of new data assets from various sources, prioritising based on criticality and value and overlap with existing data held. The ONS is considered the de facto go-to organisation for access and analysis of high-value data sets. Metadata standards are fully implemented, allowing staff to find and re-use more data than before. The IDS will form a key part of government's activities, and this is supported by the quality data it is managing. We will be working with new data from a variety of sources never used before and processing more information than ever, facilitating and overseeing automated matching within IDS.

Strategic delivery and engagement

Led by Deputy Director for Data Acquisition

Our mission

To build a comprehensive understanding of data needs and opportunities. To be able to engage with stakeholders, suppliers and analytical users with clear purpose, one voice, and to grow and sustain trusted and long-lasting delivery partnerships that can service diverse user requirements.

Our commitments

We will review the current engagement model across the ONS for both internal and external partners, to understand what is working well and how we set ourselves up for the future, ensuring focus on internal and external user needs.

We will consider outsourcing data work to strategic delivery partners who can support the achievement of joint objectives, exploring hybrid, insourcing, and outsourcing delivery models.

We will agree with our main strategic data partners and speak with "one voice" to create trusting, long-lasting relationships built around transparency and inclusion. These relationships will balance the need for innovative changes to how data is used while adhering to our standards. For example, we will pilot new ways of accessing data and will explore new frameworks to support this.

We will expand our relationships with businesses and the Government Commercial Function to exploit the potential of commercial data and to put in place value exchange, for example offering non-monetizable benefits such as corporate-social responsibilities when a private sector supplier enables access to their data.

We will take leadership roles in government communities- particularly data, analytical, engineering and architecture - to share best practice, address cross-government challenges, seize opportunities and maximise the value of data as a strategic asset in the interest of public good.

Definitions of success

There is an effective engagement model which delivers trusted partnerships for our main internal and external data delivery partners and has implemented new ways of collaborative working leading to more flexible and streamlined arrangements to meet user needs.

There is a central view of the ONS data model,demonstrating which sources can be re-used to meet new requirements and supporting prioritisation of data integration based on wider need, criticality, and value.

The ONS is recognised for its collaboration, data capability and expertise,and similarly, we proactively recognise the value of our stakeholders in contributing to overall success.

Grown at scale, our commercial data sources meet important priorities which are delivering efficiencies across government.

Outsourced data delivery outcomes where it is pragmatic and aligned with our architecture to support an increased pace of delivery.

Where we will be in three years time

Government is joined up and sharing data quickly and effectively with the ONS, using common standards and consistent governance models. User requirements are fully understood through each stage of the data workstream. The IDS has delivered data to support analysts to work on government's main priorities and the ONS has supported government to make best use of commercial data.

Public trust in data

Led by Director for Communications

Our mission

To maintain high levels of trust in the ONS while raising awareness, understanding, and support for our new data approaches by actively involving and engaging the public in our work.

Our commitments

We will be open and transparent about our data strategy, demonstrating our commitments consistently through our actions, policy and operations.

We will make sure that the structure and processes are in place to understand public appetite and opinion on data, bringing their voices into our decision-making and service.

We will actively seek dialogue and promote our work through all channels, platforms and direct to the public, ensuring that our communications are accessible and inclusive.

We will work effectively with other organisations seeking to improve public trust and understanding in data, co-ordinating where relevant to present a joined-up and consistent approach to data.

We will continuously monitor public and stakeholder feedback,ensuring learnings are collated, shared, and used to improve our approach.

Definitions of success

Our data policies and commitments will be easy to access,understand and framed in terms of their relevance and benefit to the public.

Structure and processes will be embedded which enable the public to be actively involved in the ONS's data agenda consistently and effectively.

Our approach to data will be supported by the public with trust levels in the ONS maintained, while understanding and support for the transformed data regime has increased.

All groups - including those often under-represented - will be engaged in the ONS data approach,in line with the recommendations in the Inclusive Data Taskforce Report.

The ONS is a trusted and respected data leader,modelling public involvement best practice and helping to improve standards across government and beyond.

Where we will be in three years time

The ONS will exemplify public involvement best practice through its approach to transparency, listening and engagement. The public will understand and be supportive of the transformation agenda and actively involved in its ongoing development to realise the value and full potential of data.

###Smooth end-to-end data journey

Led by Deputy Director for Data Engineering, Operations and Delivery

Our mission

To deliver a smooth, commonly understood end-to-end data journey, which streamlines ONS and cross-government working and builds trust with the public in our data.

Our commitments

We will define and publish an end-to-end journey from data collection and acquisition to publication for how data will be handled, with clear roles, responsibilities, and processes and engaging material showing what goods look like.

We will develop an automated workflow for the main stages of the data journey and generate efficiencies in data engineering and operations to speed up delivery and provide consistency.

We will have clear management information reports on DAP platform and ingestion which shows delivery progress of important datasets through the different stages of the end-to-end data journey.

We will test that management information meets needs at the earliest opportunity in the data lifecycle, for example working through seconded analysts and with new tools.

Processes and systems will be put in place to ensure effective monitoring and continual improvement to ensure that data is high quality and we are efficient; less time will be spent in the early stages of the end-to-end data journey, supporting all business areas where appropriate.

Definitions of success

Users experience a smooth end-to-end data journey where processes are automated, feedback is captured, and data is delivered more quickly and to higher quality.

A fully accessible data journey is available on our intranet, SharePoint and published to the ONS website.

Our data delivery performance is robust and transparent and can manage stakeholder expectations.

We are educating and training other parts of government and across suppliers on the trusted and ethical use of data, for example the benefits of new tools or processes to de-identify data.

Data is higher quality and we are more efficient; less time is spent in the early stages of the end-to-end data journey, supporting all business areas where appropriate.

Where we will be in three years time

The ONS will have a strong reputation for being the quickest and most trusted organisation at making data available. We will implement best practices in specifying, monitoring and improving data quality across the end-to-end data journey. We will provide thought-leadership and lead knowledge development with other government departments to support a data-driven environment.

Data acquisition and access

Led by Deputy Director for Data Acquisition

Our mission

To treat data as an essential input to our way of working and central to our future operations. To be data-driven in our production of statistics, research and data science and treat the data we acquire and access as a valuable asset for our transformation activities.

Our commitments

We will champion and evidence the benefits of data sharing with all stakeholders,matching analytic and research requirements with data-led solutions, even as things change. We will "anticipate and acquire" data in advance of need to future proof for questions we do not yet know, and to respond more quickly to crisis events.

We will create a "single source of truth"about the full suite of data sharing arrangements we have in place across the ONS and their associated legal gateways and we will improve the transparency of the data we handle and how it supports outcomes for the public good.

An Application Programme Interface (API) by default approach will be implemented across the ONS. This spans the entirety of our operations from data collection from business, data connectivity inwards and outwards with government departments, including relevant dashboards, and crucially for public consumption. This will improve the user experience across the end-to-end journey.

We will review the maximum flexibility provided by existing legislation and negotiate more flexible data sharing arrangements with streamlined stewardship to match our strategic vision.

We will work jointly with Cabinet Office to agree on critical national data,and pilot new ways of accessing data via the IDS, for example accessing data at source and not presuming that data will be physically shared with us.

Definitions of success

More organisations are sharing more datasets for a wider range of purposes,and we have implemented a new capability to access datasets at source.

There are fewer, and timelier, data sharing negotiations as we move to a model whereby data is used and reused, with streamlined stewardship.

Delivery partners across government are signed up to common principles for data access across government,using standardised governance agreements alongside flexible approaches to how data is accessed. We have agreed the ONS's critical national data, as well as identified critical national data across government, and made those available via the Integrated Data Service - fit for a culture which embraces data sharing.

A real-time view of what data is being used to support analysis and insight, which makes us the most transparent organisation in government.

A value exchange and financial efficiency from the use and reuse of data across government is delivered. We will have a mechanism to monitor and record overall cost savings and benefits to evidence the value of data and justify future investment in capability.

Where we will be in three years time

Data acquisition will be the central point for all data sharing activity across the ONS; it will have proactively acquired data in advance of need and negotiated sustainable arrangements which reflect new ways of accessing and governing data. There will be a tremendous range of new, joined-up data accessible for analysis, research, and statistical development, and a clear understanding on what data is available, leading to less frustration on accessing information. There will be a default to using APIs to share data and we will enable the public, government colleagues and stakeholders more generally to exchange data in a way which is as simple and consistent as possible.

Data quality

Led by Deputy Director for Quality and Improvement

Our mission

To ensure that quality of data is known, communicated, and managed consistently throughout its full data lifecycle supporting quality analysis and outputs.

Our commitments

We will create a culture that puts data quality at the core of every activity in the ONS.We will assess the ONS against the data maturity model to baseline our culture.

We will ensure all our critical data is fit for purpose, or that actions are in place to improve it.

We will ensure data quality will be properly managedacross the data lifecycle for all data.

We will ensure data quality is assessed at the earliest stages of planningfor services, data storage, design, and data collection.

We will embed data quality as a primary aspectin our relationships with data suppliers and as part of our engagement with them.

Definitions of success

Data quality processes are established to ensure consistency and clarity. People in all roles understand the importance of data quality and the role that they have, and they feel empowered to question and assess the quality of data they are given and to properly ensure quality in data they are responsible for.

Critical data has been identified and its quality is well understood,with consistent approaches to recording and addressing poor quality.

Governance, roles, and responsibilities for data quality management will be clearly defined,ensuring that work is consistent and coordinated across the ONS. Everyone will know and understand their role in relation to data quality with a co-ordination responsibility created to ensure that data quality is dealt with consistently across the ONS.

All new projects consider data quality during their design phase.Projects do not proceed if they have not sufficiently considered how to establish and sustain quality of data over their lifetime.

There will be a clear understanding of the quality of our data pipelines.We have an effective feedback loop to support conversations with providers on data quality.

Where we will be in three years time

We will know the quality of data, how to manage it and how to ensure it improves in the future. We will have the organisational responsibilities, commitments and processes we need to ensure that quality is properly managed throughout the data lifecycle. Awareness of data quality will be improved with greater consideration to getting data quality right from the start of the data lifecycle, with reduced reliance on the later processes to address data quality issues.

Data security, protection and ethics

Co-led by Chief Security Officer and Data Protection Officer

Our mission

To support the UK Statistics Authority's (UKSA) ambition through developing secure options that provide business expanded access to data, maintain public and stakeholder trust and confidence in how we access, use, process, store, and make available data for statistics and research purposes.

We do this by developing secure solutions that meet business objectives for data use and access in the operational context, supported by clear, easily accessible, and widely understood processes, policies and principles of data security, data protection and data ethics.

Moreover, we will be using operational protective monitoring to underpin increased data access and wider data sharing while safeguarding data, as well as using established legal and policy frameworks to uphold the principles of good data protection practice and the identification and mitigation of legal or ethical risks in the access, use or sharing of data.

Our commitments

We will understand business ambition and requirements to develop secure options that reflect these to enable business data and statistics outcomes while operating within corporate risk appetite.

We will provide a secure environment,with comprehensive protective monitoring, integrated into business area operations that enable flexible, wider use and sharing of data across the ONS and partners.

We will strengthen the ONS record management and support areas to uphold high standards of data management to reduce risk.

We will operate business-linked data security, data protection and data ethics processes to support their wider objectives, that is accessible and easy to understand. Supported by strong teams who can advise, support, and help deal with issues quickly and efficiently. We will educate staff to recognise legal, ethical and security scenarios in their work and handle these confidently.

We will work with the Cabinet Office to define a target data ownership model for government and develop a new data stewardship model for the ONS that balances sufficient oversight of data use with efficient and flexible access.

Definitions of success

Business areas using data operate securely,with audit trails and accountability for access and sharing.

The Information Asset Register is the single source of the truth for data within the ONS and supports records management across the office.

Protective monitoring covers ONS data assets and user access to these, with strong reporting links to the ONS directorates and divisions.

Data security, protection, ethics, and research governance processes are available to staff.All staff have undertaken relevant training on the safe, ethical and secure use of data and assurance frameworks are in place to ensure this is adhered to.

There is a transparent and streamlined approach to data ownership,in which decisions on data access and usage are taken quickly and efficiently, by the most appropriate steward. There is an agreed steward responsible for all integrated data assets responsible for decision making.

Where we will be in three years time

The ONS will have fully adopted the security and information strategy and have a clear view on how it can use data. Data breaches and data issues risk will be reduced, as we move over to more proactive approaches to data security and data protection. There will be close adherence to the records management system and a culture of data security across the ONS, with the benefits of data de-identification commonly understood. Utilising the expertise and skills of the Data Security and Data Protection teams, we will be supporting more teams and divisions with more complex, risk-based data acquisition and ingestion tasks. Routine legal and ethical risk assessment, identification, and mitigation "by design" at the outset and at project-level. There will be a new stewardship model for the ONS that ensure quick and efficient access to data, where appropriate.

Data capability

Led by Deputy Director, People Capability

Our mission

To ensure the ONS will have the skills to meet its current commitments and our future challenges and opportunities.

Our commitments

We will continue to assess the skills required to develop future capabilityand provide support to gain those skills, for example around Cloud Computing, Natural Language Processing, geospatial, and so on.

We will strengthen our critical workforce groups,supporting individuals to grow and develop their careers within a well-defined profession, for example Data, Digital and Technology (DDaT) Analysis Function, Government Social Research (GSR) and Government Statistical Service (GSS) professions.

We will continue to develop the organisation's data dexterity skills,establishing a common understanding of the skills, knowledge, experience, and behaviours that are critical for individual and organisation success.

We will align our recruitment and capability support to build on the BASE review findings.Learning pathways are in place underpinned by Communities of Practices for all DDaT roles with Heads of Profession.

We will continue to build our capability in technology,methods and data. For example, reviewing where modern Privacy Enhancing Technologies (PETs) and synthetic data could overcome traditional blockers to data access, while adhering to our security and ethics principles.

Definitions of success

A clear understanding of the future data skills and capabilities over time,as defined by horizon scanning the external environment and a portfolio view of the organisational strategy over the next three years.

The organisation responds flexibly to emerging learning and capability requirements of its workforce.

The learning offer from the ONS has provision to develop data skills and capabilities.

There are strong mentoring and coaching relationships within teams to help share and build knowledge about data.

Reduced our digital and data vacancies to under 10% of total DDaT headcount.

Where we will be in three years time

We will have a workforce which is motivated by data and understands the importance of a baseline set of data dexterity skills. Learning requirements have been identified and include, inter alia:

• artificial intelligence

• natural language processing

• data modelling and storytelling

• leadership

• creativity

• complex problem solving

Individuals will understand their career paths, their responsibility for building identified skills and capabilities, and will be supported in doing so by the organisation.