1. Output information

 Survey name  Opinions and Lifestyle Survey (OPN)
 Frequency  Eight months of the year (January, February, April, May, July, August, October, November)
 How compiled  Cross-sectional sample survey
 Geographic coverage  Great Britain
 Sample size  2,010 individuals per month
 Last revised  31 October 2019


Nôl i'r tabl cynnwys

2. About this Quality and Methodology Information report

This quality and methodology information report contains information on the quality characteristics of the data (including the European Statistical System’s five dimensions of quality) as well as the methods used to create it.

The information in this report will help you to:

  • understand the strengths and limitations of the data
  • learn about existing uses and users of the data
  • understand the methods used to create the data
  • help you to decide suitable uses for the data
  • reduce the risk of misusing data
Nôl i'r tabl cynnwys

3. Important points

  • The Opinions and Lifestyle Survey (OPN) is an omnibus survey that is conducted eight months of the year (two months collection, one month no collection).

  • Customers can commission questions on the survey on a variety of topics.

  • The sample size for the OPN is 2,010 individuals per month, with an average response rate of 55% (approximately 1,100 completed interviews per month).

  • The turnaround time for the OPN is a maximum of 14 weeks from finalisation of questions to data delivery.

  • Data are collected using an online self-completion questionnaire; alternatively, if required one of our highly skilled interviewers will conduct the interview by telephone.

  • Customers receive a report and data including frequency tables, standard errors, confidence intervals and supporting documentation for their commissioned questions and for demographic variables.

Nôl i'r tabl cynnwys

4. Quality summary

Overview

This report relates to the Opinions and Lifestyle Survey (OPN), which collects data from individuals (aged 16 years or over) in Great Britain.

Uses and users

The OPN is used by government departments along with charities and academia to obtain a fast turnaround of opinions data on a variety of topics each month. This diverse questionnaire collects timely data for research, policy analysis and campaign evaluation. Data are also used in the Adult smoking habits and Internet access – households and individuals publications, which are National Statistics produced by the Office for National Statistics (ONS). All OPN anonymised datasets are made available on the UK Data Service (UKDS).

Strengths and limitations

The main strengths of the OPN include:

  • it allows for a quick turnaround of data: the survey has a maximum turnaround of 14 weeks from finalising questions to the delivery of data, which allows customers to analyse timely, relevant data on topics of interest

  • it meets data needs: the questionnaire is developed with customer consultation and design expertise is applied in the development stages

  • it is flexible and responsive, allowing clients questions to be included with less notice when necessary (questions are confirmed approximately one month before data collection)

  • it meets clients’ sampling needs: questions can be run for multiple months, with the data combined to increase the sample size

  • its questions are straightforward and directed at the majority of the population, however it is also possible to include questions only relevant for sub-samples

  • it can be used to collect data on topics that are too small to warrant surveys of their own

  • questions can be included for testing purposes before they are included on other large-scale household surveys

  • robust methods are adopted for the survey’s sampling and weighting strategies to limit the impact of bias

  • it is accurate and reliable; the questionnaire is rigorously tested and the data is quality assured

The main limitations of the OPN include:

  • the monthly sample size is relatively small: 2,010 individuals per month with approximately 1,100 completed interviews per month

  • in-depth probing of topic modules is not possible due to the length of the questionnaire

Recent improvements

Over the last few years, work has been ongoing to change the design of the OPN from a face-to-face survey to a mixed mode design (online first with telephone follow-up). Mixed mode collection will allow respondents to complete the survey more flexibly and will provide a more cost-effective service for customers. Pilot research also suggests that mixed mode collection (online and telephone) will result in an increase in the number of completed surveys.

The first phase of transformation was completed in April 2018 when the OPN moved to telephone data collection. This phase of transformation included a change to the sampling frame – from the Postcode Address File (PAF) to the Annual Population Survey (APS) – and a questionnaire redesign. Specialist research, design and testing has been conducted to inform the transformation of the survey. Three pilot tests were conducted to understand the impact of changing mode and sample frame on data quality, sample representativeness and bias. Findings demonstrated minimal impact on most of the data. Where some differences were found, these were likely due to the redesigned question wording, which was optimised for telephone collection. For more information on the pilots and findings, please see the Opinions and Lifestyle Survey: mixed mode pilot analysis article.

The second phase of transformation was completed in November 2019 with the move from telephone only to mixed mode collection. From November 2019, OPN data are collected using an online self-completion questionnaire. Alternatively, if required, the interview can be conducted by telephone.

Nôl i'r tabl cynnwys

5. Quality characteristics of the Opinions and Lifestyle Survey (OPN)

This section provides a range of information that describes the quality and characteristics of the survey.

Relevance

(The degree to which the survey meet users’ needs.)

As the survey is monthly and multipurpose, collecting information on a variety of topics for the clients that commission the questionnaire modules, data are always relevant for the users.

The Opinions and Lifestyle Survey (OPN) is used for:

  • providing quick answers to questions of immediate policy interest

  • measuring the efficacy of publicity campaigns

  • measuring public awareness of new policies

  • question testing and piloting

  • testing questions on subjective well-being, including split-sample trials

  • providing a sample of respondents for other surveys (subject to respondents’ consent; personal information is never linked to survey responses)

Topics that have been commissioned by customers on the survey include smoking habits, cancer awareness, charitable giving, disability, relationships with others and internet access.

Accuracy and reliability

(The degree of closeness between an estimate and the true value.)

The total error in a survey estimate is made up of two types: sampling error and non-sampling error.

Sampling error

The OPN is a sample survey, so estimates are subject to sampling variability. Sampling variability is dependent on several factors, including:

  • the size of the sample

  • the effects of the sampling method

  • the effects of weighting

Standard errors, which give an indication of the amount a given estimate deviates from the average estimate over all possible samples, are supplied for all key variables. The method used to estimate standard errors accounts for the sample design and weighting method.

Non-sampling error

The main sources of non-sampling error are:

  • frame under-coverage

  • non-response

  • response errors (such as misleading questions or interviewer bias)

  • errors when imputing or processing data

To minimise the effects of non-sampling errors, the questionnaire is carefully designed and tested, interviewers are trained, several attempts are made to contact respondents and extensive quality control procedures are used throughout. Weighting is also used to compensate for non-response and frame under-coverage.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)

The demographic data collected on the OPN meet harmonised standards (across Government Statistical Service surveys).

Previously, OPN collection months differed to their current pattern. Prior to April 2005, the Opinions Survey was conducted for eight months of the year, with a monthly set sample of 3,000 addresses. From April 2005 until April 2014, the survey was run for all 12 months, with a set sample of 2,010 individuals each month.

The Opinions Survey was part of the Integrated Household Survey between January 2008 and December 2009, but this change did not affect the data that were deposited in the UK Data Archive.

The Opinions Survey was merged with the ONS General Lifestyle Survey (GLF) in 2012. To reflect the merger, since April 2012 the survey has been known as the Opinions and Lifestyle Survey (OPN). There was no change to the survey methodology, and so the data are comparable with the data collected on the Opinions Survey. It should be noted that comparisons with data previously collected on the GLF may vary as the survey tool has changed. The GLF was a household, annual survey with a slightly larger sample size than the OPN, which is an individual, monthly survey. After the first year of the OPN, comparisons between the data collected on the two surveys can be compared and analysed.

There are regular modules that have been running for a long period of time, for example the tobacco module. The OPN does not have a minimum number of months that customers must commission their modules for; those that are run for a short period of time would not be suitable for time series analysis. For information on how long commissioned modules were run for, please email the ONS Omnibus Service.

After the move to telephone collection in April 2018, some smoking estimates collected on the OPN and published in July 2019 experienced discontinuity. This has been acknowledged in the published datasets.

For five months in 2019, from July to November, the OPN will be combined with the European Health Interview Survey to enable both surveys to use the same sampling frame and to reduce the burden for the telephone operations team.

In November 2019, the OPN moved to mixed mode collection. This means that OPN data will be collected using an online self-completion questionnaire. Alternatively, if required, the interview is conducted by telephone. Three pilot tests were conducted to understand the impact of changing mode and sample frame on data quality, sample representativeness and bias. For more information on the pilots and findings, please see the Opinions and Lifestyle Survey: mixed mode pilot analysis article.

Accessibility and clarity

(Accessibility is the ease with which users can access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

Anonymised data are sent to the UK Data Service (UKDS). Once the data are released by UKDS, the data are available to any user, without charge, provided they have registered with the UKDS. A user guide (technical report) is provided with each OPN dataset on UKDS.

Timeliness and punctuality

(Timeliness refers to the lapse of time between data collection and data delivery. Punctuality refers to the gap between planned and actual data delivery dates.)

The OPN collects data in eight months across the year. The survey period lasts 14 weeks. This includes four weeks for developing and testing the questionnaire in Blaise (a computer-assisted survey design and processing system) and preparing derived variables. The telephone data collection period is four weeks. After the data collection period, the data are cleaned, weighted, analysed and tabulated by the research team. Data and reports are sent to clients on an agreed date, approximately five weeks after the data collection period.

The OPN provides data to clients with a rapid turnaround, but it is usually the responsibility of clients to further analyse and disseminate their findings. Occasionally, the survey is commissioned to write reports or news releases. Previous examples include reports on smoking behaviour and contraception use.

For more details on related releases, the GOV.UK release calendar is available online and provides advance notice of release dates.

Concepts and definitions (including list of changes to definitions)

(Concepts and definitions describe the legislation governing the output, and a description of the classifications used in the output.)

National Statistics Socio-economic Classification (NS-SEC)

We use the National Statistics Socio-economic Classification (NS-SEC). The NS-SEC aims to differentiate positions within labour markets and production units in terms of their typical “employment relations”.

Proxy response

The OPN is different from other ONS social surveys in that it does not collect proxy responses. This is because a large proportion of questions asked on the survey are opinions questions. These are unsuitable for proxy collection due to their subjective nature, which if collected would reduce the accuracy and quality of the data through adding missing or inaccurate responses.

Why you can trust our data

The ONS is the UK’s largest independent producer of statistics and its National Statistics Institute. The Data Policies and Information Charter, available on the ONS website, details how data are collected, secured and used in the publication of statistics. We treat the data that we hold with respect, keeping it secure and confidential, and we use statistical methods that are professional, ethical and transparent. More information about our data policies is available.

The OPN is carefully designed and tested, and extensive quality control procedures are used throughout.

Nôl i'r tabl cynnwys

6. Methods used to produce the Opinions and Lifestyle Survey (OPN) data

The data are collected using telephone interviewing over a four-week period. The objective is to collect accurate relevant data on a survey with a quick turnaround, allowing clients to receive cleaned and processed data only five weeks post collection. The population of interest is residents of Great Britain aged 16 years and over with an achieved sample of approximately 1,100 interviews each collection month.

Sampling frame

The sample for the Opinions and Lifestyles Survey (OPN) is drawn from the Annual Population Survey (APS), which consists collectively of those respondents who successfully completed the last wave of the Labour Force Survey (LFS) or the local LFS boost. Around 38,000 households respond to the LFS each quarter, and it is the largest regular household survey in the UK.

The sampling frame used for the LFS is the Royal Mail’s Postcode Address File (PAF) of small users. The PAF is the most comprehensive address database in the UK. It is updated every three months and contains approximately 27 million addresses. The sample for the LFS covers the whole of Great Britain, excluding the Isles of Scilly.

In the last wave of the LFS and LFS boost, respondents are made aware that they may be contacted for future research. The OPN sampling frame includes those individuals who have not objected to future research. This includes members of households whose final interview was completed by proxy. Of these respondents, only those who have provided a valid telephone number are included in the sampling frame, so they can be contacted to take part. As these respondents have taken part in the last wave of the LFS or LFS boost, their responses are used to reduce any potential bias and increase precision in OPN estimates.

Sampling

A two-stage approach is applied to sampling. In the first stage, a sample of households from the Annual Population Survey (APS) – those completing the last wave of the LFS or LFS boost – is drawn; in the second stage, one individual from each sampled household is selected. To overcome under-representation of younger age groups in the sample in the last wave of the LFS and the LFS boost, younger people are oversampled.

Each month 2,010 individuals are randomly selected to take part in the OPN. The selected respondent is the only household member who is eligible to participate in the survey (aged 16 and over). Proxy interviews or responses are not permitted.

Using the last wave of the LFS and LFS boost introduces a potential source of bias as it only includes those who have completed all waves of the LFS or the LFS boost and not objected to re-contact. However, data collected in the previous wave are used in the weighting process to reduce this potential bias. This is achieved by including population controls based on variables collected in the previous wave. Previous data are also used to adjust for non-response bias before weighting the adjusted sampling weights to population estimates.

The use of previous data in weighting also improves precision, although it can be slightly below that achieved under the previous design. This is because sampling from the last wave of the LFS and LFS boost, and the oversampling of younger age groups, makes the sampling weights in the new design more variable. For information on response rates please email the ONS Omnibus Service.

How we process the data

The data is validated and cleaned, variables are derived, and weights are applied. As the OPN collects information on a sample of the population, the data is weighted to enable us to make inferences from this sample to the entire population. Details about the purpose of population weighting and the weights that are supplied with the OPN datasets are provided in our methodology guide.

We produce datasets (frequencies) and calculate standard errors and confidence intervals for questions with base sizes greater than 50 cases. For questions with more than 50 cases, the data are split into various demographic breakdowns.

How we quality assure and validate the data

There are quality assurance processes from the drawing of the sample and development of the questionnaire to delivery of the final data. These processes include:

  • the accuracy of contact information in the sample is validated

  • the questionnaire is tested by both by the research team and telephone operations through test scenarios

  • post collection, a series of checks are conducted on the data to identify inconsistencies and invalid responses

  • standard errors are quality assured by comparison with frequency tables

How we disseminate the data

Customers are securely sent a report including the dataset, frequencies, standard errors, confidence intervals, questionnaire documentation, and survey information for their questions and demographic variables. Additionally, anonymised data are sent to the UK Data Service (UKDS). Once the data are released by UKDS, the data are available to any user, without charge, providing they have registered with the UKDS. A user guide (technical report) is provided with each OPN dataset on UKDS.

Nôl i'r tabl cynnwys

Manylion cyswllt ar gyfer y Methodoleg

Efallai y bydd hefyd gennych ddiddordeb yn: