1. Executive summary

In December 2018, the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined the Office for National Statistics's (ONS's) proposal to collect information on health and unpaid care.

For Census 2021, data are collected on:

-   general health

-   long-lasting health conditions and illnesses

-   activity restriction

-   provision of unpaid care

Data collected on health and unpaid care contribute to the funding, development and planning of health care and carers' services and to helping reduce inequalities experienced by those with disabilities.

The questions on health, long-lasting health conditions or illnesses, and activity restriction are:

-   How is your general health?

-   Do you have any long-term physical or mental health conditions or illnesses lasting or expected to last 12 months or more?

-   Do any of your health conditions or illnesses reduce your ability to carry out day-to-day activities?

The question on provision of unpaid care is:

-   Do you look after, or give any help or support to, anyone because they have any long-term physical or mental health conditions or illnesses, or problems related to old age?

Census 2021 will be an online-first census, with a target of 75% online returns. We have reviewed and tested each question online and have recommended additional design and functionality amends specific to the electronic questionnaire. We have also recommended additional changes to the question designs following our stakeholder engagement, evaluation and testing programmes. These changes will be detailed in this report.

The main recommended changes to question designs are:

-   the long-term health or disability question should change terminology from "health problems or disability" in the question stem to "physical or mental health conditions or illnesses" to clarify that the question asks about mental and physical health

-   the term "disability" should be removed from both health and unpaid care question designs

-   the 2011 Census long-term health or disability one-stage design should be changed to a two-stage design measuring long-lasting health conditions and illnesses and then activity restriction, which is based on the Government Statistical Service's (GSS's) harmonised principle questions

-   the time ranges in the question asking about unpaid care should be broken down into more response options, narrowing the time ranges to capture more detailed information

The Census 2021 variables will be long-lasting health conditions or illnesses and activity restriction for Census 2021 outputs.s.

Nôl i'r tabl cynnwys

2. Aim of question development report

Since the publication of the White Paper, the Office for National Statistics (ONS) has conducted and concluded the final phase of testing on health and unpaid care. This report provides links to previously published research and the findings of additional testing that led to the final recommended questions for Census 2021 in England and Wales.

The questions and response options for Census 2021 have now been finalised through the census secondary legislation: The Census (England and Wales) Order 2020 and Census Regulations for England and for Wales.

The evidence base for the recommendations made in the White Paper is discussed in Section 4: Research that led to the 2018 White Paper recommendations. The evidence base for the finalisation of the questions for Census 2021 is discussed in Section 5: Research that led to the recommended Census 2021 question designs.

Nôl i'r tabl cynnwys

3. Research and development timeline

In June 2015, the Office for National Statistics (ONS) held a formal, 12-week consultation process asking census users for their views on the topics that were required in the questionnaire in England and Wales. The aim of the consultation was to promote discussion and encourage the development of strong cases for topics to be included in Census 2021.

In May 2016, the ONS published its response to the Census 2021 topic consultation. This set out our updated view on the topics to be included in Census 2021, including:

  • a summary of proposals for new topics
  • next steps
  • an overview of our plans

The Census 2021 topic consultation revealed a clear requirement for information on general health and long-term health and disability to guide public health spending, resource allocation and policy development. There was also a clear requirement for information on unpaid care to plan, develop and deliver of a wide range of care services, including those required under the Care Act 2014 and the Social Services and Well-being (Wales) Act 2014.

A detailed summary of the consultation responses relating to the health topic can be found in the health topic report (PDF, 1.32MB). The report includes questions on general health and long-term health or disability, now long-lasting health conditions and illnesses, and activity restriction. A detailed summary of the provision of unpaid care topic can be found in the carers topic report (PDF, 606KB).

In these reports, the ONS made clear commitments to the public: we do not intend to change the questions on the health and unpaid care topics, and we will review the health and unpaid care questions' guidance, formats and wording to improve data quality or reduce burden placed on respondents.

However, preliminary work on reviewing the guidance, format and wording showed that the two commitments were not fully compatible. The 2011 Census health and unpaid care questions did not adhere to current Government Digital Service's Service Standard for the electronic questionnaire.

Also, the 2011 long-term health or disability question did not follow what is now understood to be best practice on disability measurement. The ONS therefore investigated whether to base the question for Census 2021 on the Government Statistical Service's (GSS's) service standard. This was developed between 2009 and 2011 and was published in 2011, after the last census, following broad consultation with government departments and disability groups.

Following this, we began a comprehensive programme of research and development. We provide a full list of the tests used in the development of the topics of health and unpaid care in Annex 2. Further details are provided in the Summary of testing for Census 2021.

The tests used a range of qualitative and quantitative research methods. A short description of the different research methods and sampling techniques is given in the Question and questionnaire development overview for Census 2021. Testing included respondents from a wide range of backgrounds. Respondents included people with a range of long-term health conditions and impairments and people who provide unpaid care.

In 2018, a stakeholder consultation was held considering whether to use the same long-term health or disability question as we used for the 2011 Census or whether it would be beneficial to data users to change to a question based on the GSS's harmonised principle on classification of disability in terms of the Equality Act 2010.

In December 2018, the government presented to Parliament a White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB). This outlined our proposals for the topic of health and unpaid care.

This response included commitments related to Census 2021 questions. These were to:

  • align the questions on long-term health conditions and disability with the GSS's harmonised principle questions, subject to testing
  • amend the response "Yes, 20-49 hours a week" so that the census can gather information on Carer's Allowance as requested by data users, subject to space constraints on the paper questionnaire
  • adapt all health and carer questions to follow the online Government Digital Service's Service Standard for the electronic questionnaire
  • explore improving question guidance on the health and unpaid care topics
  • investigate linking 2011 Census data to administrative health data

We have provided an update on how we met these commitments in Annex 1.

The questions for Census 2021 are now finalised. We have evaluated the question for its potential impact on data quality, public acceptability, respondent burden, financial concerns and questionnaire mode. We present details of this evaluation in Annex 3.

Annex 4 provides details of the definitions and terms used in this report.

Nôl i'r tabl cynnwys

4. Research that led to the 2018 White Paper recommendations

As set out in the White Paper Help Shape our Future: The 2021 Census of Population and Housing in England and Wales (PDF, 967KB), we proposed aligning the questions on long-term health conditions and disability with the Government Statistical Service’s (GSS’s) harmonised principle questions, subject to testing, and updating the question on unpaid care to include smaller time ranges so that information on Carer’s Allowance can be collected.

We discuss details of the testing related to the finalisation of the questions for Census 2021 in Section 5: Research that led to the recommended Census 2021 question designs.

References to tests take the form (Year: Test number). “Year” refers to the calendar year the test was undertaken in and the test number is the position of the test within the year considering all testing that took place in that year.

General health

A question on general health was asked in the 2001 Census and again in the 2011 Census. The general health question measures the current health of the population of England and Wales and helps to inform the allocation of vital resources both at the national and regional level.

The question and response options for the question on general health are unchanged from the 2011 Census. This ensures data continuity and allows comparison of data across the censuses. The general health question was tested as part of the development of the long-term illness and disability and unpaid care questions (2018:28). The question was found to be subjective, but no other concerns were raised about the question or response options.

We recommend using the 2011 Census general health question design for Census 2021.

2011 Census

How is your health in general?

  • Very good
  • Good
  • Fair
  • Bad
  • Very bad

Long-lasting health conditions or illnesses

A question on long-term health or disability was first introduced in the 1991 Census. In the 2015 topic consultation, there was a clear user need for this information to meet and monitor disability under the Equality Act 2010, statutory duties under the Health and Social Care Act 2012, developing and implementing policy, and assessing health inequalities at small area level.

2011 Census question

Are your day-to-day activities limited because of a health problem or disability which has lasted, or is expected to last, at least 12 months?

Include problems related to old age

  • Yes, limited a lot
  • Yes, limited a little
  • No

We stated in the Census 2021 topic consultation that we recommend no change to the long-term health or disability question.

However, the 2011 Census Quality Survey (PDF, 1.39MB) showed that the long-term health or disability question had a relatively low agreement rate of 88.4%. The question also created a high respondent burden by asking about health conditions and activity restriction in one question. We therefore needed to develop the question to ensure that these issues were addressed.

In addition, we began to consider using the GSS harmonised principle. The GSS harmonised principle was published in 2011, after the 2011 Census, following broad consultation with government departments and disability groups and testing. This interview-led question is used by many other social surveys, such as the Labour Force Survey (LFS). The development of the GSS’s question is discussed in further detail in Annex 5.

The GSS two-stage question was designed for interview-led purposes. Taking this question design forward, we would need to adapt it for self-completion.

Whether a two-stage design should be used

We undertook cognitive testing (2018:28) to investigate the 2011 Census design and potentially two-stage question designs for comprehension, levels of respondent burden and acceptability. A total of 31 participants took part in the cognitive interviews. Participants had a range of physical and mental health conditions, including varying age-related issues, progressive health conditions, and disability that was registered and not registered.

Three different versions of the long-term health condition or disability questions were tested:

Version 1 – the 2011 Census design:

  • Are your day-to-day activities limited because of a health problem or disability which has lasted, or is expected to last, at least 12 months? (With response options: “Yes” or “No”)

Version 2 – a two-stage design based on the 2011 Census wording:

  • Do you have a health problem or disability which has lasted, or is expected to last, at least 12 months? (With response options: “Yes” or “No”)
  • Does your health problem or disability limit your day-to-day activities? (With response options: “Yes, limited a lot”, “Yes, limited a little” or “No”)

Version 3 – the two-stage GSS’s harmonised principle questions design:

  • Do you have any physical or mental health conditions or illnesses lasting or expected to last 12 months or more? (With response options: “Yes” or “No”)
  • Do any of your conditions or illnesses reduce your ability to carry out day-to-day activities? (With response options: “Yes, a lot”, “Yes, a little” or “Not at all”)

Each participant answered a set of census questions including Version 1 of the long-term health conditions question and was then interviewed about their experience and views on the question. During the interview process, the other versions were shown to allow the participants to compare and comment on each version.

Results from testing (2018:28) indicated that, overall, participants preferred the two-stage questions to the one-stage question used in 2011 (Version 1) as this reduced respondent burden and allowed for a more accurate response.

Participants felt that the one-stage question was asking two questions instead of one and that it could not distinguish between those with no health conditions and those with a long-term condition that did not impact on day-to-day activities.

The recommendations from this test were that a two-stage question should be used and the design should be based on the GSS harmonised principle health question.

Whether the GSS harmonised principle should be used

In parallel to the initial testing, we conducted a consultation survey to understand the impact of using a two-stage question design. A survey was sent out and 316 users of the health data produced by the census responded. The aims of the survey were to understand users’ needs in more detail and investigate whether the two-stage questions met users’ needs.

The survey contained two-stage versions of the long-term health questions (Versions 2 and 3 in the initial testing phase) and highlighted the similarities and differences between them. Respondents were asked to indicate which question best met their data requirements.

Results from the stakeholder survey showed that changing to the two-stage GSS harmonised principle questions would have a positive impact for the majority of users. More than three times as many stakeholders preferred the GSS’s questions (70.30%) to the two-stage 2011 Census question (18.32%). Some users stated neither of the question designs would meet their needs, with concerns for comparability and continuity with previous censuses, but they stated there would be no impact from using the GSS two-stage design. Stakeholders also stated that using the GSS’s harmonised principle questions would mean the data can be compared against other survey data, making it more useful to them.

The recommendation from this engagement was that the design should be based on the GSS harmonised principle health question.

We undertook further evaluations on terminology to refine the recommended design for Census 2021.

Whether “mental health” and “disability” terminology should be used

Participants (2018:28) responded positively to the inclusion of mental health conditions and preferred the use of “conditions” over “problems” when talking about mental health, with “problems” thought to have a negative connotation.

Including the term “disability” was welcomed by participants with recognised disabilities and their carers. However, some thought that the term “disability” was not necessary, as it was covered by “conditions”. Some considered that the “disability” term might be less inclusive to those with milder long-term health conditions and illnesses. “Disability” was also considered to be more linked to physical than mental health conditions.

In the 2018 survey results, stakeholders reacted positively to the inclusion of mental health conditions in the GSS’s questions and reported that they were happy that the questions were now more fully aligned with the classification of disability used in the Equality Act 2010.

Some stakeholders noted that the removal of the term “disability” was in alignment with the social model of disability.

The recommendations were that the question should ask about mental as well as physical health and use the term “conditions” instead of problems and that more testing on the necessity of using the term “disability” was needed.

We investigated the implications of using the GSS’s questions further to understand if we should make this change in Census 2021.

Whether “reduce” or “limited” should be used to describe day-to-day activities

The first round of testing (2018:28) found that participants preferred the term “limited” to “reduce” when considering impact of health conditions on day-to-day activities, as it was considered to be a more understandable and clearly defined term. However, the term “reduce” is used in the GSS’s harmonised principle question, and this term is more aligned with the social model of disability than the term “limited”. To investigate which term was appropriate to use in the census, we:

  • conducted a review of the evidence from the development of the 2011 Census and the GSS’s harmonised principle question
  • considered the results of previous testing (2018:28)
  • engaged with stakeholders

Extensive testing was done on the 2011 Census question on health (PDF, 360KB), including the terms used within the question. There were no tests involving the use of “reduce” but some involving the use of “limited”. The testing found that the final question was easily understood by respondents. Additionally, participants preferred that the term “limited” was included in the response options. For example, instead of “Yes, a lot” the response option should be “Yes, limited a lot”.

Evidence from the development of the GSS’s question on activity restriction showed that using “limited” or “reduce” resulted in very similar responses. However, during the consultation into developing harmonised standards for disability data (PDF, 456KB), some stakeholders reported that the word “limited” was unacceptable as it was too negative and does not follow the social model of disability.

As a result of the review of evidence and consultation with internal Office for National Statistics (ONS) stakeholders, we recommend using “reduce” instead of “limit” in the question. The main reasons behind this decision were:

  • the GSS’s harmonised principle question uses “reduce” and the census question should be in alignment with this question
  • the term “limited” could be considered as being negative
  • the term “limited” does not follow the social model of disability

Whether the instruction to include problems related to old age be should removed

In the previous census in 2011, the long-term health conditions question had an instruction to include problems related to old age. However, the GSS’s question does not mention old age. We undertook some further analysis to understand the implications of removing the explicit reference to age-related problems.

The Annual Population Survey (APS) has used the GSS’s harmonised principle questions since 2013. We compared the disability data collected in the 2011 Census to data collected by the APS between 2014 and 2016. Results showed that a higher percentage of people aged 75 years old and older were classed as disabled using data from the 2011 Census, compared to those classed as having a disability using data from the APS. A similar proportion of people aged between 65 and 74 were classed as disabled regardless of whether using data from the APS or 2011 Census. However, a higher percentage of people in each age group under 65 years old were classed as having a disability using data from the APS compared to data from the 2011 Census.

On further analysis, the wording of the question, guidance given in answering the question and the collection methodology (the APS is collected via interviews rather than by a self-completed questionnaire) have contributed to the discrepancies between the outputs.

As a result of this analysis and previous discussions with stakeholders, and considering that the GSS’s question has had extensive testing and usage since the 2011 Census, it was decided to remove the guidance. This means that there will be no reference to old age in the questions. This also aligns with the definition of disability in the Equality Act 2010, which does not reference age.

The final recommended designs for health and disability questions for Census 2021 are shown in Section 6: Questions recommended for Census 2021.

Unpaid care

The unpaid care question was first introduced in the 2001 Census and then again in 2011, enabling users to look at the increases in the “carer” population and its implications for specific groups such as young carers aged 5 to 17 years.

2011 Census question

Do you look after or give any help or support to family members, friends, neighbours or others because of either:

  • long-term physical or mental ill-health or disability?
  • problems related to old age?

Do not count anything you do as part of your paid employment.

  • No
  • Yes, 1 – 19 hours a week
  • Yes, 20 – 49 hours a week
  • Yes, 50 or more hours a week

After the Census 2021 topic consultation, we made commitments to not change the question on “amount of care provided”. However, there was a user need to collect information on carers who were eligible for Carer’s Allowance, which would require changing the response options.

In addition, the 2011 Census question design did not meet the Government Digital Service’s Service Standard for online questions.

So, we also took the opportunity to investigate if we could make any other improvements to reduce respondent burden, improve data quality and collect more detailed information for data users. As part of this, we considered how the question compared to the GSS harmonised principle question on unpaid care (2008 to 2020).

To ensure we aligned the health and unpaid care questions, we designed and tested these questions simultaneously and changed to identical terminology.

Whether the 2011 Census question is well understood

The first round of testing (2016:5) carried out was to investigate how a possible question on volunteer work interacted with the 2011 Census question on unpaid care. Further detail on the outcome of this testing for volunteering is found in Volunteering topic research for Census 2021.

Each of the 29 participants took part in a short, online questionnaire that included a question on unpaid care and volunteering, using their own mobile device, while in the presence of an interviewer. Following completion of the questionnaire, the interviewer asked them about their experience and opinion of the unpaid care and volunteering questions.

When asked about their thoughts on the 2011 Census unpaid care question, many participants felt it was too long and complicated, as it contained text, bullet points and instructions. Some participants also felt the time ranges (for example, 1 to 19 hours) were too wide and that there should be more options with smaller time ranges. The interviews also showed that many participants confused unpaid care with other activities such as general childcare, volunteering and paid care.

The recommendations from this testing were to investigate if time ranges should be smaller by consulting with stakeholders to understand their need and to shorten the question and ensure respondents understand what should be recorded as unpaid care.

Alternative phrasings of the question to be considered

Based on the first test findings and guidance from the Government Digital Service’s Service Standard, we tested including the 2011 Census bullet points in the question stem and various shortened versions of the question.

Different versions of the question were peer reviewed (2018:15). The questions incorporated various modifications of the 2011 Census question, including:

  • “family members, friends, neighbours or others” changed to “anyone”
  • “give help” changed to “give unpaid help”
  • “look after” removed from question
  • “do not count anything you do as part of your paid employment” instruction removed

Participants were shown each version of the question on paper and asked for their views on each question and then probed further on their understanding and ease of answering for each modification. They were also asked about the acceptability of each modification.

The results showed that participants favoured the use of “anyone” over “family members, friends, neighbours or others” in the question stem. The term “anyone” was considered clearer and easier to read and understand. Participants struggled with the use of “others”; it confused participants and made participants think about caring for someone as part of a volunteering activity, reducing the quality of the data.

Participants considered the use of “unpaid” in the question to be ambiguous. For example, some participants were confused whether receiving gifts or Carer’s Allowance counted as being paid. Removing “look after” did not test well. Unpaid carers identified with “look after” rather than “help or support”, particularly those who care for family members. Participants thought “look after” along with “help or support” encompassed all caring responsibilities.

Participants gave positive feedback on the inclusion of the instruction not to count anything done as part of paid employment. They reported that it was clear and distinct from the question stem, meaning it was less likely to be missed than the term “unpaid”.

Participants also suggested shortening the question even further to include a statement at the start of the question stem regarding not including paid work, such as “Excluding paid work, do you look after...”.

The recommendations from this round of testing were:

  • not to include “unpaid” in the question stem but to have a separate instruction about not including paid employment similar to “Do not count anything you do as part of your paid employment”
  • to keep “look after” in the question stem as per the 2011 Census
  • to use “anyone” instead of “family members, friends, neighbours or others”
  • to consider a question stem that starts with “Excluding paid work”
  • that further testing should include participants who are unpaid and paid carers

Response options that stakeholders require

We consulted with stakeholders with the aim of understanding if changing the time ranges would be beneficial to them. This was in response to some requests for data on those receiving Carer’s Allowance in the topic consultation. In addition, some participants in the previous round of testing (2016:5) had suggested making the time ranges in the response options smaller, creating more response options.

We asked 43 core stakeholders to respond to an online survey asking about their data needs. The results showed a medium to high need to increase the number of response options and reduce the time ranges used in the response options. By reducing the time ranges in the question, the data collected can be better used for service provision, policymaking and resource allocation.

One main user need for increasing the number of response options and reducing the time ranges was to understand the number of carers eligible for Carer’s Allowance; the eligibility for Carer’s Allowance is 35 hours a week and over. We split the “Yes, 20 – 49 hours a week” response option into “20 – 34 hours a week” and “35 – 49 hours a week”.

Another main user need was to understand the number of people at risk of poor health and employment outcomes because of caring responsibilities; the risk increases at 10 hours a week and over. We split the “Yes, 1 – 19 hours a week” response option into “1 – 9 hours a week” and “10 – 19 hours a week”.

The recommendation from the stakeholder consultation was to use the following response options:

  • Yes, 1 – 9 hours a week
  • Yes, 10 – 19 hours a week
  • Yes, 20 – 34 hours a week
  • Yes, 35 – 49 hours a week
  • Yes, 50 or more hours a week

We also consulted with stakeholders on the redesign of the question, including:

  • using “anyone” instead of listing out the groups of people that someone could care for
  • including the bullet points of the reasons people could be cared for in the question stem
  • including the instruction “Do not count anything you do as part of your paid employment”

After evaluating the impact of these changes, stakeholders agreed with the changes.

In finalising the Census 2021 paper questionnaire designs, we found that there was sufficient space to implement all of the recommended changes on both the paper and electronic questionnaires.

Whether the question wording should align with the health questions

Based on the findings and recommendations of the previous internal peer review (2018:15), we undertook further cognitive testing on different versions of the unpaid care question. A total of 31 participants took part in this testing (2018:28), including 15 unpaid carers, 2 paid carers and 14 people with long-term health conditions or disabilities. The unpaid care question was tested alongside other questions, including the long-term health questions.

The purpose of the cognitive interview was to get further feedback on the phrasing recommended in the previous research. In addition, we investigated using “long-term physical or mental ill-health or disability”, as in the 2011 Census, or “physical or mental health conditions or illness” in line with the questions on health.

In addition, participants were asked to comment on their understanding of each version and how easy they found it to answer.

Three versions of the question were tested:

Version 1 question:

Excluding paid work, do you look after, or give any help or support to anyone?

Include

  • long-term physical or mental health conditions or illness
  • problems related to old age

Version 2 question:

Do you look after, or give any help or support to anyone because they have long-term physical or mental health conditions or illnesses, or problems related to old age?

Exclude anything you do as part of your paid employment.

Version 3 question:

Do you look after, or give any help or support to family members, friends, neighbours or others because they have long-term physical or mental ill-health or disability, or problems related to old age?

Exclude anything you do as part of your paid employment.

The response options were the same for each version of the question:

  • No
  • Yes, 1 – 9 hours a week
  • Yes, 10 – 19 hours a week
  • Yes, 20 – 34 hours a week
  • Yes, 35 – 49 hours a week
  • Yes, 50 or more hours a week

Each participant answered a set of census questions including Version 1 of the unpaid care question and was then interviewed about their experience and views on the question. During the interview process, the other versions were shown to allow the participants to compare and comment on each version.

The results showed that participants disliked Version 1 of the question because the instructions were too long. Participants preferred Versions 2 and 3 because they contained more details in the question stem and had shorter instructions.

Some participants reported that Version 3 was too long and burdensome to answer, and some questioned the use of “ill-health” in the question, which they considered an old-fashioned phrase.

Overall, participants favoured Version 2 because they thought the question:

  • started with the relevant information (“Do you look after…?”)
  • used clear and precise language in the question text
  • used appropriate descriptions (“long-term physical or mental health conditions”)
  • contained short but clear instruction text

The recommendations from this testing were:

  • include a list of conditions in the question stem, for example, “physical and mental health conditions and illnesses”
  • the shorter description of “anyone” can be used instead of the longer and more burdensome “family members, friends, neighbours or others”
  • ensure it is clear to respondents that they should only include care that is unpaid

Following this research, to align with Government Digital Service’s Service Standard, the dash in the time ranges within the response options were replaced with the word “to”. For example, the response option “Yes, 10 – 19 hours a week” was changed to “Yes, 10 to 19 hours a week”.

Whether the question should align more closely with the GSS harmonised principle

The GSS harmonised principle question on unpaid care is designed to be asked by an interviewer (not as part of a self-completed survey like the census) and is a two-stage question, first asking if unpaid care is provided by the respondent and then asking how much time is spent providing care.

There were several differences between the question developed through testing and the GSS harmonised question. We undertook research to determine if the question used in the census should more closely match the GSS question. We also needed to assess the question to ensure it met Government Digital Service’ Service Standard.

Whether the census should use a two-stage question

We reviewed other survey questions on unpaid care and found that interviewer-led questions most commonly use two-stage questions, similar to the question used by the GSS. We found that surveys that are self-complete most commonly use one-stage questions.

The mode of a questionnaire, for example, interviewer-led or self-complete and paper or online, can impact the data collected. This is particularly true of subjective questions, such as the unpaid care question. This highlighted the need to use the same questions on the paper and electronic questionnaires.

We compared the data collected in the one-stage self-complete 2011 Census on unpaid care to the data collected by the two-stage interview-led Family Resources Survey in 2016 to 2017. We found that the overall distribution of unpaid carers was similar for both surveys. The distribution of carers by sex was also similar, there was a slight difference in carer estimates by age, but it was not significant. There was no clear evidence that data quality would be impacted by asking a one-stage question instead of a two-stage question.

We recommended to continue to ask a one-stage question.

Whether the census question on unpaid care should capture data on travel time

The GSS harmonised principle question in 2018 asked respondents to include travel time in their answer. However, there is no specific mention of travel time in the Care Act 2014 and travel time is not included when calculating Carer’s Allowance. Stakeholders told us that they use the census data to allocate provisions for carers and therefore it is important to provide them with data that align with current legislation. During our extensive cognitive testing of the question designs, only one person mentioned including travel time.

The recommendation was that the question should not include any instructions regarding travel time and that only time spent providing unpaid care should be recorded. The GSS harmonised principle question for unpaid care has since been revised and no longer included travel time.

Whether the census question should capture unpaid care of less than one hour

We recommended to change the second response options to capture those doing less than one hour of unpaid care a week. This aligns with question design in other social surveys such as the Health and Lifestyle Survey and the GSS harmonised principle question. The change was supported by stakeholders, including the Equality and Human Rights Commission and the Department for Work and Pensions (DWP).

The response option of “Yes, 9 hours or less” was included in an omnibus test (2018:41). An analysis of the distribution of answers from this omnibus test showed that the data collected were similar to that of the 2011 Census. This suggested that including this response option would not have any impact on the quality of data collected.

Nôl i'r tabl cynnwys

7. Annex 1: Census commitments made on the topics of health and unpaid care

In the Census 2021 topic consultation response, the Office for National Statistics (ONS) made clear commitments to the public. We committed to:

  • not change the question on “amount of care provided” or “health topics
  • continue to measure disability in line with national guidelines currently set by the Equality Act 2010 (Great Britain)
  • review the question guidance to both improve data quality and reduce the burden placed on respondents for all health and unpaid care topic questions

However, research showed that the first of these commitments was not compatible with the others. In addition, data user needs progressed since the 2015 topic consultation. As such, we later committed in the 2018 White Paper to align the questions on long-term health conditions and disability with the Government Statistical Service’s (GSS’s) harmonised principle questions, subject to testing, and to update the question on unpaid care to include smaller time ranges.

Aligning the question on health conditions and disability with the GSS’s harmonised principle questions

We tested the GSS’s harmonised principle questions and consulted with stakeholders to ask them their views on using the GSS’s questions. We concluded that using the GSS’s questions would be beneficial to both data quality and our stakeholders’ needs, so we will use these questions on Census 2021.

Updating the question on unpaid care to include smaller time ranges

We have increased the number of response options to this question to allow more detailed information on respondents’ time spent providing unpaid care. The new time ranges allow us to capture data on those eligible for Carer’s Allowance.

Nôl i'r tabl cynnwys

8. Annex 2: Summary of research undertaken for the health and unpaid care topics, 2016 to 2020

References to tests take the form (Year: Test number). “Year” refers to the calendar year the test was undertaken in and the test number is the position of the test within the year considering all testing that took place in that year. For example, the fifth test conducted in 2017 would be referenced as (2017:5).

A full description of each of these items can be found in Summary of testing for Census 2021.

Nôl i'r tabl cynnwys

9. Annex 3: Question evaluation

Evaluation of general health question

Potential for impact on data quality, public acceptability, respondent burden and financial concerns were all evaluated as “Low”.

Potential for impact on questionnaire mode: “Medium”

The general health question differs between online and paper. The question is classified as a sensitive topic and has a potential impact to be answered differently between the two modes.

Evaluation of long-lasting health conditions or illnesses question

Potential for impact on financial concerns was evaluated as “Low”.

Potential for impact on data quality: “Medium”

Feedback from the 2019 Rehearsal (2019:15) showed that respondents found this question difficult to answer on behalf of another person and that respondents used the context-sensitive help more than was expected.

Potential for impact on public acceptability: “Medium”

The long-lasting health conditions or illnesses question is a sensitive topic and as such, respondents may answer in the way they think they should be perceived in society. During the 2019 Rehearsal (2019:1), and user experience (UX) testing (2019:15), there was some negative acceptability feedback for this question. As such, for Census 2021 we have added a guidance accordion for the question on the electronic questionnaire to mitigate and provide clarity on what we are asking.

Potential for impact on respondent burden: “Medium”

Although the question has been updated to include mental as well as physical health, increasing the length of the question, testing (2018:28) showed that respondents reacted positively to the inclusion of reference to mental health conditions and illnesses. However, feedback from the 2019 Rehearsal (2019:15) showed that respondents were spending more time (13 seconds) than expected on this question and it was above the expected threshold of 10 seconds.

Potential for impact on questionnaire mode: “Medium”

To improve public acceptability and data quality, for Census 2021 we have added a guidance accordion for this question on the electronic questionnaire only. Guidance text cannot be added to the paper question because of space constraints. This difference in mode means there is a potential for impact on this question.

Evaluation of question on activity restriction

Potential for impact on public acceptability, respondent burden and financial concerns were assessed as having a “Low” potential for impact.

Potential for impact on data quality: “Medium”

The activity restriction question is a sensitive topic and is personal to individuals. The sensitive nature and the information collected from respondents may not be observed, particularly when another person is answering on proxy for another person. In the 2019 Rehearsal (2019:15), respondents on average took 13 seconds to answer this question, which is above the average of all questions.

Potential for impact on questionnaire mode: “Medium”

To improve public acceptability and data quality, for Census 2021 we have added a guidance accordion for this question on the electronic questionnaire only. Guidance text cannot be added to the paper question because of space constraints. This difference in mode means there is a potential for impact on this question.

Evaluation of unpaid care question

Potential for impact on public acceptability, financial concerns and questionnaire mode were assessed as “Low”.

Potential for impact on respondent burden: “High”

The question length has been increased to include more instruction text within the question instead of underneath it, as testing showed that respondents tended to skip over instruction text (2016:15). In addition, the time ranges used have been narrowed, meaning there are more response options. Although respondent burden has been increased by moving the instructions into the question stem, the question has also been simplified by removing “family members, friends, neighbours or others” and replacing it with “anyone”. Testing (2018:41) showed that the new question design was well understood by respondents.

Potential for impact on data quality: “Medium”

The number of response options has increased, allowing for more accurate reporting of time spent caring for others, including the ability to gather information on Carer’s Allowance, which was a stakeholder requirement. We have also changed the wording of the question to include instructional text, so it is clearer to respondents what kind of unpaid care should be counted. Despite these improvements, the question remains as having a “Medium” potential for impact on data quality.

Nôl i'r tabl cynnwys

10. Annex 4: Definitions and terms used in this report

There are two main models of disability: the medical model, which considers a person to be disabled based solely on the impairment they have, and the social model, which considers a person to be disabled based on environmental or social factors, such as lack of accessible toilets or prejudice within society and not the solely result of a person’s impairment. The UK government encourages the use of the social model of disability.

It is the Equality Act 2010 that gives a disabled person rights in law.

The guidelines for the Equality Act 2010 (PDF, 693KB) state that, in general, a person is considered to be disabled if they have an impairment that is physical or mental and the impairment has substantial adverse effects that are long-term and affect day-to-day activities.

This definition of a disabled person recognises that a person may have an impairment that is long term but may not affect day-to-day-activities or an impairment that can affect day-to-day activities but is not long term. It is usually the combination of both long-term impairment and adverse effects on the ability to carry out day-to-day activities that means a person is recognised by law as being disabled.

This report uses the definition of disability from the Equality Act 2010. The questions used in the census, which are based on the Government Statistical Service (GSS) harmonised principles, were developed primarily using this definition. Using the same definition that gives disabled rights in law is vital to the users of the data provided by the census.

Nôl i'r tabl cynnwys

11. Annex 5: The GSS’s harmonised question on long-term health conditions and disability

In line with the Government Statistical Service’s (GSS’s) harmonisation principles, the GSS developed a question designed to capture data on long-term illnesses and conditions. The question asks, “Do you have any physical or mental health conditions or illnesses lasting or expected to last for 12 months or more?”.

The GSS also developed a follow-up question designed to capture data on activity restriction caused by long-term health conditions for respondents who answer “Yes” to the previous question. This question asks, “Do any of your conditions or illnesses reduce your ability to carry-out day-to-day activities?” with response options of “Yes, a lot”, “Yes, a little” and “Not at all”.

The harmonised principle questions were developed by the GSS using guidance from sources including:

Evidence from these sources supported the separation of the concepts of health conditions, impairment and disability.

The questions were specifically designed to reflect the definition of disability in the Equality Act 2010, so the questions do not use the term “disability” but instead ask about physical or mental health conditions or illnesses. Aligning the questions with the Equality Act 2010 meant the data collected from these questions could be used to monitor legal obligations to people with disabilities.

Nôl i'r tabl cynnwys