Targeted incentives on the Opinions and Lifestyle Survey

In governmental surveys, including Office for National Statistics (ONS) social surveys, younger age groups, and ethnic minority groups, tend to be less likely to participate. This increases non-response bias and makes the sample less representative. One strategy to encourage response is to use incentives, particularly in a targeted approach, to encourage participation while maintaining cost-effectiveness.

In May 2025, we conducted a pilot study to explore offering £10 incentives to a target group consisting of those aged 16 to 29 years and/ or those in underrepresented ethnic minority groups. The pilot was conducted over six waves of the Opinions and Lifestyle Survey (OPN) to increase response and sample representativeness.

Our main findings are:

• overall response rates on OPN significantly increased by 3.5 percentage points, from 45.2% to 48.7%
• response from the targeted group significantly increased by 10.8 percentage points, from 25.3% to 36.1%
• for those aged 16 to 29 years, response significantly rose by 11.5 percentage points, from 22.0% to 33.5%, and for ethnic minorities it rose by 10.1 percentage points, from 28.7% to 38.8%
• response for ethnic minorities aged under 30 years showed the largest increase by 12.9 percentage points, from 17.7% to 30.6%
• when looking at sample composition, out of all respondents, offering £10 has increased the proportion of the targeted group by 2.8 percentage points, from 10.1% to 12.9%, which will be expected to reduce bias in the achieved sample
• after accounting for underlying changes over time, the total achieved responses for the target group increased by 5.6 percentage points compared with the non-target group (white and aged 30 years and over)
• the main recommendation from this trial is to implement this approach as a part of the regular process for the Opinions and Lifestyle Survey (OPN), subject to ethical and budget approvals

In the last few decades of survey research, response rates have faced a downward trend, and this has continued to be problematic for governmental surveys where estimates are used for policy making. The Survey Nonresponse Trends and Fieldwork Effort in the 21st Century: Results of an International Study across Countries and Surveys article, published by Luiten, Hox and De Leeuw in 2020, investigated trends in household survey non-response from 1998 to 2015. The study covered 25 countries and seven longitudinal surveys, including the Labour Force Survey and surveys relating to income and living conditions, national travel, and social-cultural changes. Although trends varied between countries, response rates were found to have continued to decline on all types of surveys, with an average 0.59 percentage point decrease for each year.

One tool to increase survey participation is monetary incentives, where respondents are offered a financial reward to encourage participation by showing appreciation for their efforts. The impact of incentives can be mixed, and it depends on when the incentives are offered and the value of the incentive.

There are two main types of incentives:

• conditional – offered contingent on survey completion
• unconditional – offered before survey completion, usually in an advance letter

Research has shown that incentives can have a positive impact regardless of when they are offered, though they only work up until a certain amount. This is discussed in the Incentive Payments on Social Surveys literature review, published by Simmons and Wilmot, in 2004; and also in The Effect of Large Monetary Incentives on Survey Completion article (PDF, 120KB) published by Hsu and others in 2017.

Therefore, this evidence suggests that an incentive of any amount can be effective at increasing the response, when compared with no incentive, and the incentive does not necessarily need to be of high monetary value. However, it should be large enough to acknowledge the survey burden and make it worthwhile for a respondent to take part.

Incentives have been shown to be particularly effective for groups who are less likely to respond, as shown in The Effects of Promised Monetary Incentives on Attrition in a Long-Term Panel Survey article (this page includes pay-to-access content), published by Zagorsky and Rhoton in 2008; and in The effect of increasing financial incentives in a panel survey article, published by Laurie in 2007. These articles show the use of differential incentive strategies, which involve offering higher value incentives to those less likely to respond, which reduces non-response bias. Furthermore, targeted incentives are a more cost-effective strategy, rather than offering monetary incentives to all sample members who may participate regardless of the incentive.

Targeted incentives have been mostly implemented in the US since the late 1990s, as shown in The use of incentives to recruit and retain hard-to-get populations in longitudinal studies article (PDF, 159KB), published by Nicolaas, Corteen and Davies in 2019. The adoption of such designs in the European and UK context is more recent. Implementation of targeted incentives is not common in government surveys, with additional considerations needed around ethics. The evidence on the impact of targeted incentives on improving response rates and sample representativeness is limited and mixed. Therefore, in this research, we explored the impact of monetary incentives targeted to underrepresented groups on the Opinions and Lifestyle Survey.

The Opinions and Lifestyle Survey (OPN) is a monthly, cross-sectional sample survey for residents of Great Britain who are aged 16 years and over. The survey covers topics relating to people's experiences of daily life and events. The survey also asks about what people feel are important issues, their health, well-being and experiences of increases in the cost of living. It regularly includes questions on a range of other topics, such as attitudes towards climate change and artificial intelligence. Further context about the survey can be found in our OPN quality and methodology information report.

Data collection is conducted through an online self-completion questionnaire; telephone interviews are available if requested by a respondent, but the predominant mode of collection is online.

From June 2024, the OPN has monthly data collection and outputs. Data collection is split over two 12-day survey periods each month. This allows data collection to be spread throughout the whole month (rather than the majority of responses occurring at the start of the month).

Since July 2024, the OPN has offered an unconditional A6 notepad sent with the invite letter. No monetary incentives are currently provided.

Sampling

From June 2024, the OPN sample is primarily drawn from respondents to the Transformed Labour Market Survey (TLFS). Details about how respondents are initially sampled for TLFS can be found in our Transformed Labour Force Survey — user guidance methodology. The secondary sample source for OPN consists of previous respondents to the OPN itself. A detailed overview is available in our OPN quality and methodology report.

Since respondents participated in TLFS and opted in to take part in further Office for National Statistics (ONS) surveys, participant characteristics, such as age and ethnicity, are already available. This allows us to target incentives to specific individuals.

Demographic data collected for each household that responded to the TLFS are used to sample individuals for OPN to form a representative sample of Great Britain. Using a sub-sample of TLFS and OPN respondents introduces a potential source of sample bias, as it only includes those who have already completed the TLFS or OPN and have opted in to follow-up contact.

Looking at the composition of OPN respondents, some demographic characteristics are underrepresented. We consider the achieved sample to be less representative if certain groups that answer survey questions differently are less likely to be represented among respondents. When comparing OPN respondents with population estimates for Great Britain, we found that, as of April 2025:

• younger age groups (aged 16 to 29 years) are the most underrepresented age group

• older age groups (aged 50 years and over) are the most overrepresented age group

• ethnic minority groups (including Asian, Black and Mixed) are underrepresented

• white ethnic groups are overrepresented

To mitigate against potential non-response bias in the OPN outputs, existing survey data are used to adjust for non-response bias before applying the adjusted sampling weights to population estimates. Information on the OPN weighting strategy can be found in our OPN quality and methodology report.

A theory of change (ToC) was developed in 2024 to map out the use of differential incentives, identifying important activities, outcomes and outputs.

The main outcomes and impacts were to find ways to increase response of underrepresented groups to improve the quality of statistical outputs by improving the representativeness of Office for National Statistics (ONS) surveys.

There are numerous ethical and legal considerations around the use of differential incentives.

Firstly, there is some concern among research ethics committees, survey sponsors, and survey providers, that giving incentives to some and not to others could be perceived as unfair. For this trial, offering incentives only to underrepresented groups was identified as a potential form of positive discrimination. However, it can also be argued that treating everyone equally does not necessarily achieve equity.

In June 2019, a workshop funded by the Economic and Social Research Council (ESRC) considered the ethics of using differential incentives in survey research, where Office for National Statistics (ONS) colleagues also attended. The main conclusion was that differential incentives are not inherently good or bad. Whether their use is acceptable depends on:

• the importance of including rarely heard groups

• the reasons why these groups are less likely to take part

• evidence that differential incentives will achieve the desired objective of reducing non-response bias and/or improving inclusiveness

The ESRC report on whether differential incentives should be used was used to inform our decision initiate this trial.

For this project, we obtained ethical and legal approval, both from the ONS and from the National Statistician's Data Ethics Advisory Committee (NSDEC). The NSDEC Board approved the trial, stating "differential incentives may be acceptable to improve survey participation among the hard-to-persuade subgroups". We took the following steps to ensure ethical compliance throughout the trial:

1. We obtained approval from the ONS ethics and legal team.

2. We submitted a paper to the NSDEC to secure wider government approval.

3. The ONS Media Communications team set up keyword alerts on social media to maintain awareness of any negative media coverage about this project, the OPN, or its incentives, during the trial period.

4. We briefed the Survey Enquiry Line (SEL) so that staff were prepared to answer questions if respondents called to ask about the survey or its incentives.

5. We added wording on the OPN survey webpage for some transparency during the trial.

The wording on the OPN survey page mentioned in Point 5 has now been removed. It previously read:

"On some occasions, we may reward people differently for taking part in our surveys. We do this because: 

a. This approach helps us collect data from a wide range of individuals. 

b. It helps us improve the quality of our statistics, which inform decision making that goes on to have a big impact on us all."

Following the completion of the trial, we reviewed SEL and social media tags. There was no negative feedback from respondents about the OPN or the trial itself.

Descriptive analysis

To explore the impact of the £10 targeted incentive, we produced descriptive statistics that compared response rates and demographic composition of respondents. We compared six survey waves that were collected before the trial with six survey waves collected during the trial period. The pre-trial period covered February to April 2025 and the trial period covered May to July 2025.

Response rates were calculated relative to the entire sample, and relative to each subgroup. Subgroup-specific response rates allow for consistent comparisons over time, minimising the influence of changes in the size or demographic makeup of the initial sample.

Logistic regression

We then conducted logistic regression to summarise the patterns in the data and estimate the impact of the intervention. The binary outcome of the logistic regression was whether or not an individual responded to the survey. The model included indicators for group membership (target or non-target), time period (pre- or post-intervention), and an interaction between these variables. This allowed comparison of response rates across four distinct groups:

• pre-trial target group, which included people who were aged 16 to 29 years and/or from an ethnic minority

• pre-trial non-target group, which included people who were aged 30 years and over, and white

• post-trial target group, which included people who were aged 16 to 29 years and/or from an ethnic minority

• post-trial non-target group, which included people who were aged 30 years and over, and white

The following regression model specification was used:

• Logit (E(p_ijt)): The logit link function applied to the expected probability that an individual, i, responds to the survey, in group j, at time t.

• β₀: The intercept term, representing the baseline level of the outcome (reference group: pre-trial non-target group).

• intervention_j: A binary variable indicating whether group is part of the target group who received the £10 voucher.

• post_t: A binary variable indicating whether the time t is before or after the intervention was applied.

• intervention_j * post_t: An interaction term capturing how the change over time differs between the target and non-target groups.

• δ : The coefficient on the interaction term, representing the additional change in response likelihood for the target group after the intervention.

The analysis also controlled for ethnicity and age (not shown in the formula) as the likelihood of an individual responding may also vary by these characteristics. This helps to remove this variation from random sampling and focus on any impact from the intervention.

The fixed effects model estimates the log-odds of the outcome variable for each of the predictor variables. For ease of interpretation, the results of the interaction terms are also presented as predicted response rates so that they can be compared with the observed response rate.

Evaluation design: difference in difference

We conducted an impact evaluation using a quasi-experimental design. We applied a difference-in-difference (DiD) method to estimate the impact of the incentive intervention on achieved responses.

To examine the impact of targeted incentives on sample representativeness, we ran a quasi-experimental trial to target younger age groups and ethnic minority groups on the Opinions and Lifestyle Survey (OPN) from May to July 2025. The younger age group included those aged 16 to 29 years. Ethnic minority groups included those who did not select a "White" ethnic group in the Transformed Labour Force Survey (TLFS).

When scoping out the research design, a randomised controlled trial (RCT) was considered. However, this was ruled out because the OPN targeted sample size meant that the time needed to run a full trial would not be operationally viable. Therefore, a quasi-experimental design was chosen, which enabled us to establish the effects of the targeted incentive within the budget and time constraints available.

Everyone in the sample received an unconditional notepad, which was the existing incentive on the OPN. The targeted group (those aged 16 to 29 years old and/or those who identified themselves as being in underrepresented ethnic minority groups) was also offered a £10 conditional incentive. The non-target group (those aged 30 years and over, whose ethnicity is white) who acted as the "control" group, only received the notepad. Therefore, the demographic composition of the target and non-target group was different.

DiD assumes that, in the absence of the intervention (for example, targeted incentives), the proportion of the target group within the sample would have remained constant over time. Figure 1 illustrates this assumption that the unobserved counterfactual should remain constant. Figure 1 also shows the assumption that the intervention has affected the outcome if the difference between the two groups changes from pre-trial to post-trial. The outcome of the DiD analysis refers to the proportion of the target group among total achieved responses.

Figure 1: Illustration of the assumption that the unobserved counterfactual should remain constant

The effect of the intervention is reflected in the DiD calculation as follows:

• Y^T​_post: average outcome for the target group after the intervention
• Y^T​_pre: average outcome for the non-target group before the intervention
• Y^C_post: average outcome for the target group after the intervention
• Y^C_pre: average outcome for the non-target group before the intervention

If the assumption that the difference between the target and non-target groups remains constant (without the presence of an intervention) is violated, estimates of the causal effect will be biased. However, as the target and non-target groups differ in their demographic composition, there is some unavoidable uncertainty about whether external influences might affect the two groups differently. To address this concern, we examined pre‑intervention trends of the target and non-target groups.  

Figure 2 shows that the difference between the composition of the target and non-target groups in the sample remained constant before the intervention. Therefore, we assume that the increase in representation of the target group among the total number of responses in each wave after the introduction of the £10 incentive can be attributed to the intervention.

Figure 2: Representation of the target group increased in the survey after the incentive trial began

Percentage of total achieved respondents from the target and non-target groups for the six waves, before and after the £10 incentive was offered

Before evaluating the impact of this quasi-experimental trial, it is important to identify whether any other trials were already in place, and if they may have affected demographic groups differently. For example, the OPN began sending reminder letters to the entire sample from 19 February, which explains an increase in response rates that week for both target and non-target groups (Figure 3). This trial remained in place for the remainder of the study and is therefore assumed not to have affected the demographic groups differently.

Research questions

During this project, we asked the following two research questions:

Research Question 1 — What is the impact when offering a £10 conditional incentive to underrepresented groups on response rate on the Opinions and Lifestyle Survey (OPN) survey?

Research Question 2 — What is the impact when offering a £10 conditional incentive to underrepresented groups on sample representativeness on the OPN survey?

We present our findings for each research question in the following sections.

Research Question 1: Comparison of response rates pre- and post-trial 

Figure 3 is evidence that the subgroup-specific response rates between the target and non-target groups are parallel before the intervention. The target group response increases directly after the intervention, showing the positive impact of the trial as anticipated with the difference-in-difference (DiD) evaluation approach. The non-target group does not observe a substantial change over time. Post-intervention, the trend remains parallel across the two groups, with a smaller difference. Increasing the response rate of the target group and reducing the difference with the non-target group was the desired impact of the intervention, illustrating increased engagement in the survey among the target group.

Figure 3: Response rates for the target group increased after the £10 incentive was offered

Figure 4 shows the statistically significant increase in overall response rate by 3.5 percentage points after offering £10.

Among the targeted group (16-to-29-year-olds and ethnic minorities), there was a 10.8 percentage point increase post-trial, compared with pre-trial response rates.

For the younger group (16 to 29-year-olds), there was an increase of 341 respondents, or 11.5 percentage points, when people in this group were offered £10.

For ethnic minorities, response rates increased by 10.1 percentage points, resulting in 186 more respondents after the £10 incentive was received.

For the small number of individuals (approximately 85 sampled per wave) who satisfied both conditions of the target group (young and ethnic minority), there was a 12.9 percentage points increase in response rate.

There was also a statistically significant increase in the non-target group, which suggests some other influencing factor(s). While it is significant, it is clearly a much smaller increase than is seen in the target group, which shows the strong positive impact of the incentive.

Figure 4: Post-trial increases in response rates for the overall group and the target subgroups are statistically significant

Logistic Regression

The logistic regression supports the descriptive findings and provides evidence that the £10 incentive increased response among the targeted groups. The model controlled for age and ethnicity and compared response patterns before and after the introduction of the incentive.

Table 1 shows the regression results. The pre-trial non-target group serves as the baseline against which all other groups are compared. Before the intervention, the target group had a 0.72 lower likelihood of responding than the non-target group. Following the introduction of the £10 incentive, the target group's likelihood of response increased significantly by a factor of 1.6. Under the assumptions in the difference-in-difference design, the large increase observed among the target group shows that the intervention was effective in increasing participation among younger people and ethnic minority groups.

The logistic regression model also predicted the probability of response for each individual in the dataset based on our predictors in Table 2. The predicted probabilities are then aggregated for each group. The averages are presented as percentages, which gives predicted response rates. The observed and predicted response rates are compared in Table 2 to validate the model fit.

Table 2 shows that the target group predicted response rates increased dramatically after the trial, as expected. The predictions were, as expected, extremely close to the observed response rates, showing a good model fit that can accurately predict whether an individual will respond to the survey. The confidence intervals based on the model are very small (less than 0.5%), confirming a high level of certainty that the trial had a substantial positive impact.

Research Question 2: Comparison of sample composition pre- and post-trial

We conducted further descriptive analysis to see whether the sample composition had changed following the trial, and to observe its impact on representativeness of the achieved sample. Figure 5 shows that offering a £10 incentive has increased representation of the targeted group as a percentage of the total number of respondents. Of all respondents, the targeted group has increased by 2.8 percentage points, which we expect would help address the previously observed underrepresentation in these groups. Of all respondents, representation of those aged 16 to 29 years significantly increased (up by 2.3 percentage points), as did representation of ethnic minority groups (up by 1.1 percentage points). The young and ethnic minority group increased by 0.5 percentage points, however, the difference was not significant.

Figure 5: Offering £10 improved representation of the target group, as a percentage of the total number of respondents

Percentage of total achieved respondents from each group before and after the £10 incentive was offered

We have looked at this both in terms of change and in comparison, with published mid-year estimates (MYEs) or Census 2021 estimates. Figure 6 shows that offering £10 to those aged 16 to 29 years brings the composition of respondents (8.3%) significantly closer to the proportion of 16 to 29-year-olds in the 2024 mid-year estimates (21.2%). The proportion of those aged 30 years and over decreased because of an increase in the younger age group, which reduced bias in the achieved survey responses.

Figure 6 also shows that offering £10 to ethnic minority groups brings the unweighted composition of respondents (5.9%) closer to the composition of the population (18.3%), according to Census 2021. The unweighted proportion of white respondents decreased because of an increase in the ethnic minority group, bringing the achieved sample closer to the population composition.

​Figure 6: Offering £10 improved the representation of the target groups, which brought the sample composition closer to the composition of the 2024 mid-year population estimates and Census 2021 estimates

Effect of incentive on representation of target groups, target group of those aged 16 to 29 years, compared with the 2024 mid-year population age estimates, and target group of ethnic minority groups, compared with Census 2021 ethnicity estimates

Difference-in-difference calculations

After comparing descriptive statistics summarising the composition of the sample by target and non-target group, the DiD calculations in this section estimate the causal effect of the incentive. The DiD calculations used the following formula to compare the composition of respondents from the target and non-target group in the sample pre- and post- intervention:

DiD calculations were based on the values presented in Figures 5 and 6, showing the total number of achieved responses, and the proportion of these responses from the target group.

After accounting for underlying changes over time, the DiD analysis showed that the percentage of achieved responses for the target group increased by 5.6 percentage points compared with the non-target group. This indicates that the £10 incentive increased representation of the target group within the overall achieved sample, relative to the change observed in the control group.

Similarly, when comparing the change in composition of the achieved sample between the target and non‑target groups before and after the intervention, the DiD analysis found that:

• for those aged under 30 years, there was a 4.6 percentage point difference

• for ethnic minorities, there was a 2.2 percentage point difference

• for ethnic minorities who are also aged under 30 years, there was a 1.0 percentage point difference

The input values for the change in composition of the achieved sample and the full DiD calculations can be found in Table 3.

Given that the trends in composition of respondents from each group remain reasonably stable over time, we assume that the parallel trends assumption is not violated.

So far, the DiD analysis has shown the positive impact of the trial on sample representativeness. The regression coefficients in Table 1 also show the impact of the intervention on the likelihood of responding to the survey. The interaction coefficient provides a DiD estimate for the causal effect of the trial, after accounting for both:

• baseline differences between target and non-target groups
• underlying changes over time affecting both groups

The interaction coefficient (DiD estimate) shows that the £10 incentive was associated with a 60% higher likelihood of response among the target group. This is over and above any change observed in the non-target group (odds ratio equals 1.60, p-value is less than 0.001).

The DiD calculations and regression analysis both account for underlying change over time between the target and non-target groups, whereas the simple descriptive analysis earlier in this section do not. Therefore, these DiD results are a more conservative estimate of the causal effect of the intervention compared with the descriptive statistics.

As the DiD results align with the conclusions of the descriptive statistics, we may conclude that the evidence presented in this article reliably shows that targeted incentives improve representation and response rates among underrepresented groups.

Limitations

Since the OPN draws on a sub-sample of the Transformed Labour Force Survey (TLFS) and previous OPN respondents who have consented to further research, this introduces selection bias in the sample. Additionally, TLFS respondents who are then sampled for OPN only complete Wave 1 (out of 5 waves) of TLFS and receive £10 for doing so. OPN respondents may therefore expect to receive an incentive for completing the OPN survey and choose not to respond, given that monetary incentives are not usually offered. Therefore, it is difficult to fully conclude that the targeted incentives are the reason for increase in response in a targeted group, as targeted respondents may have already been expecting a reward, given their recent experience of completing the TLFS.

In future, we may want to test targeted incentives with the longitudinal TLFS. By giving higher incentive amounts to underrepresented groups on Wave 2 onwards and non-responders from Wave 1, targeted incentives may further improve participation. This further research may give a better reflection on the true impact of targeted incentives for longitudinal survey contexts.

This pilot study provides evidence that targeted monetary incentives can create a positive impact on response rates and sample representativeness within the Opinions and Lifestyle Survey (OPN). Offering a £10 incentive to younger people and ethnic minority groups resulted in a statistically significant increase in overall response rates by 3.5 percentage points. More notably, response rates among the targeted subgroups rose by 10.8 percentage points post-trial. The highest gains in response rates were observed among people in ethnic minority groups aged under 30 years (up by 12.9 percentage points) and people aged 16 to 29 years (up by 11.5 percentage points). These improvements translated into a 2.8 percentage point increase in the proportion of targeted respondents within the achieved sample, contributing to a reduction in non-response bias.

The recommendations from this trial are to implement targeted incentives as a regular part of the OPN process, subject to ethical and budget approvals, which are currently in progress. We would also like to explore wider applicability across our flagship surveys where feasible. This could include Wave 2 onwards of the Transformed Labour Force Survey and potentially Wave 2 onwards of the Wealth and Assets Survey, subject to further assessment of suitability, operational impact, and budget and ethical considerations. Target groups may include non-responders of Wave 1 and underrepresented population subgroups. Such approaches may yield deeper insights into the true effectiveness of targeted incentives and inform strategies to improve inclusivity and data quality in the context of longitudinal surveys.

Incentive

A form of compensation for a respondent's time and effort to fill in a survey. Incentives can be monetary and non-monetary and offered either ahead of survey participation (unconditionally) or after the respondent has taken part in the survey (conditionally). Incentives are also used to increase survey responses.

Randomised controlled trial

A form of research trial where individuals are assigned randomly to different experimental conditions. There are usually one or more experimental conditions, where the impact of the experimental conditions is compared with a control condition that has no intervention applied.

Cross-sectional survey

A survey in which respondents are asked to take part at one specific point in time. It provides a snapshot of the views and experiences of a population at a particular moment, rather than following the same individuals over time.

Quasi-experimental study

A research method that tests the effect of an intervention but does not use random assignment to create comparison groups.

Theory of Change

A Theory of Change is a planning tool that explains how a programme or intervention is expected to achieve its goals. It is often described as a "roadmap" because it sets out the pathway from activities to outcomes, showing the steps, assumptions and conditions needed for change to occur.