Internet access - households and individuals QMI

This quality and methodology report contains information on the quality characteristics of the data (including the European Statistical System five dimensions of quality) as well as the methods used to create it.

The information in this report will help you to:

• understand the strengths and limitations of the data

• learn about existing uses and users of the data

• understand the methods used to create the data

• help you to decide suitable uses for the data

• reduce the risk of misusing data

• The Internet access survey measures household Internet access and how the internet is used by adults in Great Britain

• The survey data are collected in interviews conducted as part of the Opinions and Lifestyle Survey in Great Britain.

• The results are published annually in the Internet access – households and individuals Statistical bulletin.

Overview

This report relates to the internet access survey estimates, which are calculated from data collected in modules of the Opinions and Lifestyle Survey (OPN) in Great Britain. The survey aims to provide users with information on households’ and individuals’ use of the internet and other information and communication technologies (ICTs). The estimates have been published annually since 2006.

The OPN is an omnibus survey, which is conducted eight months across the year (two months collection, one month no collection). The Internet access data are collected in the January, February and April modules of the OPN.

ONS first collected statistics on Internet access in 1998. Since then, various changes have been made to the collection and publication of Internet access statistics, including the publication of annual results since 2006. Where possible, the data tables show comparisons over time, however, time series comparisons included in tables vary as the survey questions change from year to year to reflect changes in the use of technology and use of the internet.

The primary publication method is the Internet access - households and individuals Statistical bulletin and related data tables. This is published annually in August.

ONS conducts a separate Internet Access survey for Northern Ireland, asking the same questions that are used in the OPN interviews in Great Britain. The dataset from the Northern Ireland survey is combined with data collected by the OPN in Great Britain. This enables UK results to be prepared and supplied to Eurostat in accordance with EU requirements. Eurostat publish these results, allowing comparability with other countries.

From April 2018, the OPN data collection process changed from face-to-face interviews to a telephone only survey. Further details of this are given later in this report.

Uses and users

The policy-owning department for Internet access is the Department for Digital, Culture, Media and Sport (DCMS). DCMS’s departmental plan includes a priority to “continually drive the UK’s connectivity, telecommunications and digital sectors” and it has also set out the aim of “leading the digital revolution to make the UK the most competitive and innovative market in the world and driving an ambitious digital communications agenda, connecting homes and businesses to broadband and mobile”. Our internet access estimates help progress towards these priorities to be monitored.

Eurostat are also a key user and the results enable international comparability between the UK and other countries. The supply of results to Eurostat is currently an EU legal requirement.

Strengths and limitations

Strengths

One of the main strengths of the Opinions and Lifestyle Survey (OPN) is that it allows for a quick turnaround of data; the survey has a maximum turnaround of 14 weeks from finalising questions to the delivery of data. Another strength are the methods that are used from the start of the questionnaire to the delivery of the data. Design expertise is applied in the questionnaire development stages, alongside customer consultation to ensure data needs are met. Robust methods are also adopted for the survey’s sampling and weighting strategies to limit the impact of bias and to provide high quality data. Accuracy and reliability are also assured through rigorous testing of the questionnaire and quality assurance processes.

The survey design of the OPN allows it to carry various modules covering a range of subject areas. The survey is run for eight months spread throughout the year, and the Internet access module is run on three of these. Some of the Internet access survey questions change each year to measure changes in the use of the internet and new technology and the OPN is easily able to accommodate these changes.

Limitations

The main limitation of the OPN is its sample size, which is relatively small compared with other ONS social surveys. This resulted in a net sample size of 2,754 for the Internet access results in 2018. The OPN is designed to be a sample survey of the population of Great Britain, but not lower levels of geography and for this reason, as well as because of the small sample size, there is limited scope for results to be prepared at regional levels.

Recent improvements to the Opinions and Lifestyle Survey

Over the last few years work has been on-going to change the design of the OPN from a face-to-face survey to a mixed mode design (online first with telephone follow-up). Mixed mode collection will allow respondents to complete the survey more flexibly and will provide a more cost-effective service for customers. Pilot research also suggests that mixed mode collection (online and telephone) will result in an increase in the number of completed surveys.

The first phase of transformation was completed in April 2018 when the OPN moved to telephone data collection. This phase of transformation included a change to the sampling frame and questionnaire redesign.

In April 2018 the sampling frame changed from the postcode address file (PAF) to the Annual Population Survey (APS), which consists collectively of those respondents who successfully completed the last wave of the Labour Force Survey (LFS) or the local LFS boost. Using this sampling frame allows potential respondents to be contacted by telephone. In the last wave of the LFS and LFS boost, respondents are made aware that they may be contacted for future research. The OPN sampling frame includes those individuals who have not objected to future research. This includes members of households whose final interview was completed by proxy. Further details of the new sampling frame and sampling approach can be found in the OPN methodology guide.

Specialist research, design and testing has been conducted to inform the transformation of the survey. Three pilot tests were conducted to understand the impact of changing mode and sample frame on data quality, sample representativeness and bias. Findings demonstrated no significant differences for the internet access variables analysed. For more information on the pilots and findings, please see the Opinions and Lifestyle Survey: mixed mode pilot analysis article.

The OPN plans to move to mixed mode collection (online with telephone follow-up) in November 2019.

This section provides a range of information that describes the quality and characteristics of the data and identifies issues that should be noted when using the output.

Relevance

The Internet access survey collects data on households’ access to, and adults’ use of, the internet. Some of the questions change each year to obtain new, relevant information, on different topics relating to the use of the internet and technology.

Within the UK, there is wide interest in these statistics from government as well as researchers, public bodies, charities and academics. The UK also provides statistics on households’ access to and individuals’ use of the internet to Eurostat, to allow comparability between the UK and other EU countries. Eurostat also compiles EU indicators which help monitor the development and use of ICTs and help to provide a better understanding of the adoption of ICTs and the internet by households and individuals

Accuracy and reliability

The Internet access results, as with other sample surveys, are subject to error consisting of two elements: the sampling error and the non-sampling error.

Sampling error

The Opinions and Lifestyle Survey (OPN) is a sample survey, so estimates are subject to sampling variability. Sampling variability is dependent on several factors, including the size of the sample, effects of the sampling method and effects of weighting. Standard errors, which give an indication of the amount a given estimate deviates from the average estimate over all possible samples, are supplied for all key variables. The method used to estimate standard errors accounts for the sample design and weighting method.

Non-sampling error

The main sources of non-sampling error are frame under-coverage, non-response, response errors (such as misleading questions or interviewer bias) and errors when imputing or processing data. To minimise the effects of non-sampling errors, the questionnaire is carefully designed and tested, interviewers are trained, several attempts are made to contact respondents and extensive quality control procedures are used throughout. Weighting is also used to compensate for non-response and frame under-coverage.

Coherence and comparability

From 2006 onwards, the data from three separate monthly Internet access data collections each year have been combined to produce the dataset for the annual Internet access statistical bulletin. Before 2006, the Internet Access Survey did not cover Northern Ireland but in 2006, coverage of Northern Ireland was introduced via the Northern Ireland Omnibus Survey, to enable UK estimates to be produced. As a result, the annual Internet access – households and individual’s statistical bulletins were published on a UK basis for the years 2006 to 2010.

Coverage of Northern Ireland as part of the Northern Ireland Omnibus survey ceased in 2008 but for 2009 and 2010, UK results were produced by estimating results for Northern Ireland. In 2011, when there appeared to be no prospect of the resumption of the collection of Internet access data in Northern Ireland, the process of estimating for Northern Ireland was discontinued and the results were published in the Internet access bulletin, for the first time, on a Great Britain basis. In 2015, coverage of Northern Ireland was resumed in a stand-alone telephone survey, to enable UK results to be supplied to Eurostat. The Internet access statistical bulletin did not include these results and therefore remains on a Great Britain basis.

Most of the Internet access estimates take the form of proportions of households or adults. As Northern Ireland only comprises approximately 3% of the UK adult population, the proportion estimates are usually the same, whether the coverage is Great Britain or UK. When there have been differences, these have been in the order of only one percentage point.

Accessibility and clarity

Our recommended format for accessible content is a combination of HTML webpages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. Our website also offers users the option to download the narrative in PDF format. In some instances, other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information please refer to the contact details at the beginning of this report.

For information regarding conditions of access to data, please refer to the following links:

• Terms and conditions (for data on the website)
• Copyright and reuse of published data
• Pre-release access (ended from 1 July 2017)
• Accessibility
• Access to microdata via the Secure Research Service (SRS)

In addition to this Quality and Methodology Information, basic quality information relevant to each release is available in the quality and methodology section of the relevant annual Internet access statistical bulletin.

Timeliness and punctuality

The Internet access data are collected by the OPN in January, February and April. The OPN survey period lasts 14 weeks. This includes four weeks for developing and testing the questionnaire in BLAISE (a computer-assisted survey design and processing system) and preparing derived variables. The telephone data collection period is four weeks. After the data collection period, the data are cleaned, weighted, analysed and tabulated.

The estimates are published in the Internet access – households and individuals statistical bulletin on our website in August of the year that the data are collected. EU aggregates are published around the end of the year on the Eurostat website.

For more details on related releases, the GOV.UK release calendar provides 12 months’ advance notice of release dates. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Statistics.

Concepts and definitions

The Internet Access survey is harmonised across EU member states and meets EU regulation requirements to allow comparability of results published by other countries. The specific data requirements of the survey are defined annually through a consultation process whereby users and policy makers within Eurostat discuss requirements and proposed questionnaire changes are discussed in meetings with Eurostat and EU member states. Once the details of the survey requirements have been agreed, an annual implementing regulation is passed to turn the requirements into EU law.

We take part in this annual process and fully engage in the development of the questionnaire for each year. This is to ensure, as far as possible, the survey results will be useful and that respondents selected for the survey will be able to understand the survey questions.

The survey includes questions relating to purchasing goods and services online (e-commerce). Questions relating to this topic use the Organisation for Economic Cooperation and Development (OECD) definition of E-commerce. This is to ensure that e-commerce activities are measured in a consistent and comparable way across countries. OECD defines an e-commerce transaction as:

“the sale or purchase of goods or services, conducted over computer networks by methods specifically designed for the purpose of receiving or placing of orders.’”

Underpinning this definition is that:

“the goods or services are ordered by those methods, but the payment and the ultimate delivery of the goods or services do not have to be conducted online.”

Geography (including list of changes to boundaries)

The data collected in the three OPN modules (January, February and April) give a sample size large enough to provide good quality results for Great Britain, but the sample is not large enough for detailed regional results. In the past, the statistical bulletin has included limited regional estimates; however, the sample size is not sufficient to continue to do so. All the published results for 2018 therefore relate to Great Britain.

A sample size large enough for detailed regional estimates would require the Internet access module to be run in additional months of OPN. However, the timetable for publishing the results on our website in August and preparing the results for submission to Eurostat by the October deadline would make this difficult. The required funding for collecting the Internet access module in additional months would also need to be available.

Why you can trust our data

ONS is the UK’s largest independent producer of statistics and its national statistics institute. The Data Policies and Information Charter detail how data are collected, secured and used in the publication of statistics. We treat the data that we hold with respect, keeping it secure and confidential, and we use statistical methods that are professional, ethical and transparent.

Weighting

Weights are used in the analysis of the survey data, for both households and individuals, to produce the results. Respondents are assigned an individual and a household weight so that the total of all their weights sums to the adult population and whole population, respectively, in Great Britain. The individual weights are calculated using the characteristics of responding individuals only, whereas the household weights are calculated using the characteristics of all members of the household. These weights are derived by calibration, using population estimates for age group by sex and region. The weights improve the accuracy of results by compensating for different response rates for different groups and by reducing the random variation in estimates. For the household analysis, the weights are influenced by the characteristics of all the members of the household.

Weights used in the Opinions and Lifestyle Survey (OPN) dataset adjust for the unequal probability of selection caused by interviewing only one adult per household and oversampling of younger people.

Despite the efforts made by interviewers to maximise response rates, approximately 40% of selected individuals do not take part in the survey. Differential non-response among main subgroups in the population is especially problematic because it can result in biased estimates. Consequently, respondents belonging to sub-groups that are prone to high levels of non-response are assigned higher weights. For example, young males living in London have a lower response rate and are therefore assigned higher weights than males living in other regions. Weighting respondent data by age, sex and by region to estimate population totals reduces the standard errors of survey estimates if the survey variable is correlated with age, sex and region.

As the data are weighted to population totals, this means that weighted totals of individuals by age group, sex and region from the survey are guaranteed to match the fixed population totals. However, the weighting does not include control totals for the total number of households. Previous Internet access – households and individuals publications, up to 2012, contained estimates of the numbers of households with Internet access, but the numbers of households shown in the data tables for these publications were survey estimates and subject to random variation.

National Statistics Socio-economic Classification

The National Statistics Socio-economic Classification (NS-SEC) replaces previous classifications that were based on social class and social and economic group.

Proxy response

The OPN does not collect proxy responses.

Further information about the Opinions and Lifestyle Survey is available on our website.

How we disseminate the data

Our primary method of disseminating the data is through our Internet access - households and individuals statistical bulletin. Results on a UK basis are also supplied to Eurostat. The publication of the results by Eurostat enables comparison of the UK with results for other EU member states. Bespoke special analyses can also be commissioned on request and these will also be published on our website.

Contact details for this report

esociety@ons.gov.uk