Cognitive testing of loneliness questions and response options

Many different approaches have been used to measure loneliness. These include both loneliness multi-item and single-item measures. Some measures ask about loneliness directly while others ask about emotions associated with loneliness, from which loneliness is then inferred. There are advantages and disadvantages associated with each. We undertook a programme of scoping work and consultation with experts on existing approaches to loneliness measurement. From this, two preliminary measures (both a direct and an indirect measure of loneliness) were selected as meeting many criteria we required for the loneliness indicators.

However, before making a final recommendation on the measures, cognitive and survey testing of our preliminary recommended measures of loneliness was conducted. This was intended to provide further information on how they would work for people of different ages and backgrounds and how well they would perform on different types of surveys. The main findings from the survey testing have been reported in the Testing of loneliness questions in surveys chapter, while this chapter focuses on our cognitive testing work. Our cognitive testing involved qualitative interviews in which respondents were first asked to answer the proposed loneliness questions, followed by discussion of their interpretations of the questions and use of the response scales. As the questions were already in use among adults, the cognitive testing focused on how well the questions would work with children and young people.

In particular, this chapter outlines:

• the questions and response options that were tested

• the methodological approach for the cognitive testing

• the findings from the cognitive testing for children (aged 10 to 15 years)

• the findings from the cognitive testing for young adults (aged 16 to 24 years)

• children’s and young adults’ preferences for response categories

• children’s and young adults’ preferences on where they would complete these questions

• children’s and young adults’ opinions on the impact these questions would have on survey respondents

The findings and recommendations in this chapter were used to inform our recommended national indicators and the guidance for measuring loneliness in national surveys.

Following an initial scoping review and short-listing of loneliness measures with experts, we cognitively tested four questions to capture different aspects of loneliness. The first three questions were from the UCLA three-item loneliness scale, which is currently used in the English Longitudinal Study of Ageing and the last is a direct question about how often the respondent feels lonely, currently used on the Community Life Survey.

2.1 Young adults

These questions were tested with young adults aged 16 to 24 years:

1. How often do you feel that you lack companionship?

2. How often do you feel left out?

3. How often do you feel isolated from others?

4. How often do you feel lonely?

Response categories: Hardly ever or never / Some of the time / Often

2.2 Children and young people

An adapted version of the measures was tested for use with children and young people aged 10 to 15 years. The wording for the children’s measure was changed to a more “plain English” version, reflecting concerns that the words “companionship” and “isolation” are difficult for children to read and may be interpreted in a range of different ways. We revised the questions and tested them qualitatively (to understand children’s ease of use and interpretations) and on a survey of children conducted by The Children’s Society.

The following questions were tested with children and young people aged between 10 and 15 years:

1. How often do you feel that you have no one you can talk to?

2. How often do you feel left out?

3. How often do you feel alone?

4. How often do you feel lonely?

Response categories: Hardly ever or never / Some of the time / Often

Children’s and young adults’ understanding of the questions, the meanings of the words and concepts, and the suitability of the response options were cognitively tested. The cognitive testing was the first part of a semi-structured interview, which focused on respondents’ understanding and experiences of loneliness. The interviews lasted approximately 60 minutes for young adults (aged 16 to 24 years old) and around 30 minutes for children and young people (aged 10 to 15 years old). The cognitive part of the interview asking about understanding of the questions lasted for approximately half of the interview.

3.1 Approach to sampling and recruitment

Recruitment of respondents took place during July 2018. The main sampling criteria were agreed and monitored throughout recruitment of study respondents to try to achieve a balanced sample.

Several methods were used to recruit respondents including asking Office for National Statistics (ONS) to circulate information about the research to people with children and young people aged 10 to 24 years (see Annex 1). Recruitment was also carried out in collaboration with children’s charities, namely The Children’s Society and Whizz-Kidz. The latter acted as an intermediary, passing on information about the research to potential respondents on our behalf, while researchers from The Children’s Society collaborated with ONS throughout and were actively involved in the design of study materials, respondent recruitment and interviews.

The sample design sought an even balance between males and females, age groups, region, and rural and urban areas. Other characteristics such as ethnicity and disability were to be monitored to help ensure a mixture of views and experiences among respondents. The achieved sample had slightly more females than males and more respondents living in urban than rural areas. The age breakdown of respondents included an even mix of children and young people in two age groups (sixteen children aged 10 to 11 years and fifteen young people aged 12 to 15 years) and young adults, also in two age groups (sixteen young adults aged 16 to 18 years and sixteen aged 19 to 24 years).

All respondents were given a token of appreciation to thank them for taking part. These were in the form of a £15 high street store voucher for those aged 10 to 15 years and £30 in cash for those aged 16 to 24 years.

3.2 Achieved sample

In total 63 interviews were completed. Table 1 provides a breakdown of the characteristics of the respondents.

Most interviews were conducted during August 2018 in the participant’s own home, with six conducted in a youth centre and four at ONS premises. The interviews were predominantly completed face-to-face with no one else present although several younger children were interviewed with a parent or guardian present.

3.3 Topic guides, recording and transcription

The interviews were based on a topic guide used as an aide memoir that also allowed flexibility for responding to topics raised by respondents. Small changes were made to the topic guide as the research progressed in order to make the interview flow more smoothly and to enable questions emerging from previous interviews to be addressed. A copy of the final topic guides can be found in Annex 2 and Annex 3.

In keeping with best practice, all interviews were recorded with permission of the respondents (and their parents where appropriate) and transcribed word for word. Prior to turning on the recorder, respondents were reminded of the reason for the interview and what would happen with their information and that the findings from the study would be reported anonymously with their data held confidentially. They were also informed that they could stop the interview at any time and that they did not have to answer any questions that made them feel uncomfortable.

Children and young people along with a parent or guardian were asked to read and if in agreement sign a consent form (Annex 4). Parents and guardians in conjunction with their children were then asked if they would like to be present during the interview. A similar process was conducted with young adults and they were asked to sign a consent form, though their parents or guardian were not present (Annex 5). A copy of the signed consent form was left with the respondents so that they could contact the interviewer at a later date if required.

Our analysis, as described in Section 3.4, is based on the transcribed data.

3.4 Approach to analysis

The qualitative data from the cognitive interviews were analysed in a multi-stage process:

• immediately after the interviews the interviewers wrote up the main themes based on the original topic guide; slight revisions were then made to the topic guide to incorporate any emerging themes to be explored in subsequent interviews

• regular meetings were held between interviewers and other members of the research team to share and explore emerging initial themes

• the transcribed interview data were used for the full thematic analysis, with the thematic framework developed collaboratively by the research team

• to generate early themes, the same transcript was analysed by several members of the team to ensure a consistent approach

• finally, all transcripts were analysed using the agreed thematic framework

This report presents the findings from part of the interview data involving cognitive testing of respondents’ interpretations of the loneliness questions and possible alternatives. Substantive findings about children and young people’s experiences and perspectives on loneliness have been published separately.

This section presents the findings from cognitive testing of children and young people (aged 10 to 15 years), focusing on their understanding and interpretations of each of the four loneliness questions.

4.1 “How often do you feel that you have no one to talk to?”

4.1.1 Understanding of the question

Children interviewed generally showed good understanding of the question, “How often do you feel that you have no one to talk to?”.

A common interpretation was that the question was asking if they had someone to talk to about their feelings and issues important to them in particular:

“So basically, how often is there no one around which you can trust to say stuff to. […] Yeah because you won’t tell someone everything who you’ve just met. Like you won’t go I live at blah, blah to a complete stranger who you’ve never met before.” (Male, 12)

Similarly:

“How often do you feel like you can go to someone and express how you feel and your emotions and what you’re thinking about?” (Female, 15)

The “double negative” aspect of the question could be challenging to interpret, as highlighted by the need for a respondent to re-phrase it aloud before answering:

“I always have someone to talk to, so that’s hardly ever.” (Male, 15)

The lack of a specified timeframe for the question could also be problematic:

“[…]trying to remember because it’s really hard to remember back all the way to first school, like year 1 and Reception.” (Male, 12)

4.1.2 Alternative interpretations

We found some variation in children’s understanding of what it means to have someone to talk to. For some, the question simply asked if they had anyone to talk to at all:

Interviewer: “How often do you feel that you have no one to talk to, can you tell me what you think that question is asking you?”

Child: “How often is there people around to talk to?” (Male, 14)

4.1.3 Understanding of “companionship”

We tested the “How often do you feel that you have no one to talk to?” question in place of the UCLA original item “How often do you feel that you lack companionship?” due to concerns that younger children may have difficulty understanding the concept of “companionship” or reading the word. Our testing supported this view, as there was variation in whether children aged 10 and 11 years had ever heard the word, or knew what it meant. Among those who understood the concept of companionship, there was little consensus on its interpretation. A companion could be synonymous with a friend or family member:

“Like your friends like they’re your companion, I guess.” (Female, 14)

“Is it like a companion where you’re together or something, like your friends or family?” (Male, 12)

Alternatively, companionship could imply a closer relationship than just someone to talk to. For example, a companion was considered to be someone they could rely on to be there for them and who they felt comfortable with. This view was more common among older children:

“Companionship sounds more like a closer relationship than just someone there, because you can really, if you feel comfortable with them, talk to them. But I’m not best friends with my form tutor or my parents, so companionship’s maybe more of like a closer bond.” (Female, 14)

Another interpretation was that it referred to relationships with pets. It is possible that is interpretation has its roots in children’s stories that often include animal “companions” and may be where many children first encounter the concept of companionship.

“Like a friend, someone that you do a lot of things with[…] or like a pet, a dog.” (Female, 11)

4.1.4 Preferred question wording

Respondents who preferred the revised version (“How often do you feel that you have no one to talk to?”) felt it was easier to understand:

“[I would prefer the no one to talk to question] because if they’re like my age but slightly older, sometimes people aren’t quite sure, like they think they know what it means slightly but not fully. So, I’d probably keep [how often do you feel you have no one to talk to].” (Female, 11)

Similarly:

“Well I think the first one [about no one to talk to] because I think the second one's [companionship] a bit too vague. Because lacking companionship, like some people might not exactly understand what that means, like lacking companionship does that mean that you don't have friends or that you don’t spend time with your friends or that you don’t feel that you like your friends or like being around your friends.” (Male, 14)

They also believed that other children would have no difficulties understanding and responding to the question, “How often do you feel that you have no one to talk to?”.

4.2 How often do you feel left out?

4.2.1 Understanding of the question

Children commonly had a good understanding of the question, “How often do you feel left out?” and had no difficulty responding. They understood being “left out” as meaning excluded or marginalised in relation to group activities or relationships. For example:

“It’s where they don’t want to play with you, you’re not allowed to play and you’re sitting in a corner and then ten people are just playing over there, football, and they won’t let you play.” (Male, 12)

“Left out in my opinion would be that you're with a group of people and they've sort of gravitated towards each other but away from you. So, you're sort of left on your own, on your little island, the figurative island and away from them. And I guess you're sort of isolated because people, they probably won't to talk to you because they're talking to each other. And yeah, I think that's a good explanation of it.” (Male, 14)

4.2.2 Alternative interpretations

Being left out could also be interpreted as social exclusion in a wider sense:

“How often do you feel like you have no place, or like serve no purpose or feel isolated within society?” (Female, 15)

Considering why people are left out, it could be either be something imposed by others or it could be self-imposed:

“So like when you’re with your friends, how often do they leave you out the group, how often do they not let you join in with what they’re doing or talking about?” (Female, 11)

“[…]If you tell yourself oh they don’t want me there or they don’t want me to do that, then you can get yourself in that mindset and be like I’m not going to go because I know they don’t want me. But also, people can also shut you out and not let you do stuff with them and deliberately leave you out.” (Female, 14)

When discussing what being left out feels like, it was very much about being alone with no social support:

“Well when you’ve got no one, you’ve got no friends or you’ve got no one there to talk to, and you’re just always by yourself.” (Female, 11)

4.3 “How often do you feel alone?”

4.3.1 Understanding of the question

Children also showed a good understanding of the question, “How often do you feel alone?”, and had little difficulty answering the question. Again, understandings varied. The question could imply how often you feel by yourself:

“When you have no-one really or, yeah, you just feel alone[…] You could be in a group, but they don’t include you. You could feel lonely, but you wouldn’t be alone.” (Female, 14)

It could also mean being around people who aren’t engaging with you:

“Alone means there’s people around you but they aren’t talking.” (Male, 11)

4.3.2 Alternative interpretations

Although all children were asked about “feeling” alone, some misunderstood or interpreted this as “being” alone:

“I’d probably think of alone as being by yourself.” (Female, 11)

“Like how do you feel when you’re by yourself, not a lot of people around you, socialising.” (Female, 10)

Being alone was not always seen as a negative thing, and there was some indication that time away from other people provided a way to get desired privacy or a way to calm down:

“Yeah because you want a moment to yourself, some privacy, yes.” (Male, 12)

“You can be alone by yourself if you have got into an argument and you want to calm down, you can be alone.” (Female, 10)

Choice was an influential factor when deciding if being alone was a positive or negative thing. Young people felt that being alone by choice could be a positive experience, but that being alone without choosing was negative:

“I think that sometimes it's good to be alone. You don’t want to constantly be surrounded by people. But I think it should be a choice. You don’t want to be alone without wanting to be. It's not a very nice feeling.” (Female, 13)

A distinction was also drawn between being left out and being alone, with being left out leading to being alone if you had no one else to go to:

“Being left out is just being left out of one thing, but then you can find something else to do; whereas being alone is like you’ve got no one else to go to if you’ve been left out of something.” (Female, 11)

4.3.3 Understanding of “isolated”

Again, we tested the “How often do you feel alone?” question in place of the UCLA original item “How often do you feel isolated?” due to concerns that younger children may have difficulty understanding the concept of “isolated”. This concern was shared by respondents who felt that some children would not understand the word “isolated”:

“Some people might not fully understand what isolation is. They might feel like it's something different than it is. But most people will know what alone means and lonely, so again it's more straightforward.” (Female, 13)

“Because isolated is quite a more dramatic word than alone. So, they might not understand it or they might just like, I don’t know how to explain it, but they might just like feel more sad.” (Female, 10)

“Isolation” appeared to have a wider range of interpretations than “alone”. For example, isolation was seen as more extreme or severe than being alone:

“If you’re isolated it’s like you’re on your own and no one else can get to you. It’s a bit different to alone I think. Because sometimes you can feel alone even though there’s other people around you or people like there, you just maybe make yourself feel more alone because you want to be by yourself instead of forced to be by yourself, maybe isolated is more[…]” (Female, 14)

Similarly:

“[…] isolated I feel it's a much harsh term. Like isolated could imply that you feel that there's no one around you, that no one wants to talk to you, there's no way you'll talk to anyone. And yeah I think that alone and isolated probably on the same spectrum but I think isolated is more extreme than alone.” (Male, 14)

Isolation was also associated with punishment in schools such as when a child is separated from their classmates and sent to do their work alone somewhere else:

“It means when you’re alone. Because we have isolation at school where you’re put on a table by yourself outside the office, so you don’t distract other people. […] Alone, by yourself, no one.” (Male, 11)

This connection to punishment could contribute to the perceived severity of the word “isolated” among children.

Another interpretation of isolation was that of self-isolation and withdrawing oneself from others:

“I think alone because there are very few people cut themselves off from your world because people generally do have friends and family to talk to. So being alone I think is more common than being isolated.” (Female, 14)

“To me, if you're isolated, it means that you're not only being like left out you've also got to the point where you're pushing people away or isolating yourself.” (Female, 13)

4.3.4 Preferred question wording

When asked whether young people felt it better to ask “How often do you feel alone?” or “How often do you feel isolated?”, “alone” was preferred over “isolated” due to concern that other children would not understand the question and the belief that “isolation” was a far more severe experience.

4.4 How often do you feel lonely?

4.4.1 Understanding of the question

Children understood the question well, but felt that it was similar to or the same as, “How often do you feel alone?”.

“When you have no-one really or, yeah, you just feel alone[…]You could be in a group, but they don’t include you. You could feel lonely, but you wouldn’t be alone.” (Female, 14)

This also highlights a clear understanding of the difference between feeling alone and being alone, the former being a state of mind, regardless of the physical presence of others:

“Like how often do you feel like by yourself at home, at school, anywhere round where you are by yourself, no one talking to you, just by yourself, no one but there’s a lot of people around you.” (Male, 13)

Respondents also noted that a defining aspect of loneliness was a negative feeling:

“Lonely is just like you’re by yourself but you’re sad. But alone is just you’re by yourself” (Male, 13).

4.4.2 Alternative interpretations

Another aspect of “feeling lonely” is that it could be seen as a more prolonged and potentially damaging emotional state than “feeling alone”:

“How often, I feel like alone is like a short period of time you can feel alone, but then lonely is like a long period of time. So, you can feel lonely for a long time but alone is more like you feel alone for a day and then you can kind of snap out of it or you can kind of go get better again and, you know, do something.” (Female, 14)

“I’d say being alone would be a much more present feeling, something you feel right now; being lonely would be something a lot more prolonged. I think lonely would be where you’ve spent so much time alone that it’s started to have a negative effect on your mental wellbeing […] where it starts to feel negative, where you start to think, I don’t like this” (Male, 14)

A total of 31 young adults between the ages of 16 and 24 years took part in similar cognitive interviews as those reported in Section 4 with children and young people. The questions and response options tested with this group were the same questions used on the English Longitudinal Study of Ageing (ELSA) and Community Life Survey (CLS), although the latter survey uses a five-item response scale. The CLS also includes this age group as part of its adult general population sample but only has the direct question on loneliness. This testing enabled further insights into how young adults in the UK respond to the UCLA questions, first developed among university students in the United States.

In general, the four questions were answered by the 16- to 24-year-old respondents without any difficulty. In the following sections we focus on interpretations of each question.

5.1 How often do you feel you lack companionship?

5.1.1 Understanding of the question

Respondents of this age generally understood the question about lacking companionship in terms of lacking friends or friendship, but a range of interpretations were offered as to the depth and nature of the relationship implied by the word “companionship”.

Respondents thought that companionship meant having someone they could talk to, who would understand them and offer support when required:

“It’s sort of like if I was having a bad day or if I was, you know, if I needed someone to talk to I would have access to someone to talk to, like a friend or a close family member who I could look at and look for help from.” (Female, 23)

“Well I suppose it’s probably asking if you, no matter where you are if you feel like you don’t have any company or someone to talk to. You’re just, doesn’t really matter who they are it’s just if you have someone who you’re with or you can speak to, or like a friend or family. And I suppose yeah, makes sense yeah.” (Male, 19)

They also noted that this was not a word they would normally use, which may help to explain why there were varying interpretations as to the type of relationship implied by it:

“I laughed because we would never say that. Why would you? You wouldn’t say it.” (Female, 18)

“It depends. Everybody's got a different definition of companionship. What one person would consider companionship is going to be different from what somebody else considers it to be.” (Female, 23)

“Companionship to me is like a relationship between two people in a way, or multiple people. Like, there's a bond there between them. It could be friend. It could be romantic. It can be anything.” (Male, 16)

5.1.2 Alternative interpretations

Companionship could be interpreted as a more superficial friendship, potentially lacking depth or closeness:

“So I see it in terms of being close to someone. You can have companions but still like kind of be[…] it’s a bit like glass between the friendship and it doesn’t actually feel like you know them well. Then you’ll have people you know who you feel very warm around – more intimate – relationships where you feel like very comfortable. You know you can be companions with someone and still feel like you lack companionship if that makes sense.” (Female, 23)

“Companionship is probably more open, because at least if you have a companion you can at least try and build on that relationship[…] you just meet someone. You wouldn’t say you were friends with someone immediately after you meet them. But then you might go to have a drink at Costa or something. That’s a companion.” (Male, 18)

It could also imply a more romantic or intimate relationship. This was mentioned by respondents of both sexes from across the age range:

“I would say it’s more like probably like dating life and stuff like that really especially with my sort of age.” (Male, 23)

“I guess you could look at it in terms of general friendship maybe also in the sense of a relationship like a more intimate kind of companionship. That’s sort of how I see it.” (Female, 23)

This theme also carried through into a sense of discomfort that the word could be interpreted as a romantic relationship and should be avoided for that reason:

“Maybe change the word companionship, because some people can take that as an affectionate, so you've got a companion in your life could be like wife, girlfriend, boyfriend, stuff like that.” (Male, 16)

“I suppose I feel like perhaps especially, more men would feel a bit, maybe the word companion would feel a bit flowery or[…] I feel like I know that a lot of my male friends probably wouldn’t say they would have a companion because while you have a great relationship, you can have a really great relationship with another guy, it’s kind of, I think calling them your companion wouldn’t feel right.” (Male, 17)

The term “companion” was also interpreted in quite different ways by disabled respondents. A companion could be someone who provides care or support with daily activities (as in companion care) or alternatively, a companion may be a friend, but not a paid carer:

“A lot of people think of, especially in the disability sector, companionship is relying on someone else[…] in the disability sector companion is quite often a negative connotation because it’s kind of like a carer. So for me companion makes it feel like I rely on someone, whereas actually rather than being supported by someone.” (Female, 21)

“Well do you lack a person there that’s not helping, not designed to help you. Basically somebody who’s not there as a helper, but more as a friend I suppose[…] Somebody who isn’t a carer. Because carers, as nice as they are, are paid to be there, they’re there to support you, and they shouldn’t really be used as a substitute.” (Male, 22)

5.1.3 Understanding of “no one to talk to”

We also compared interpretations of the companionship question with the “plain English” adaptation of the question used with children and young people (“Having no one to talk to”). Respondents commonly felt that this was more straightforward and easier to understand:

Respondent: They’re the same[…]

Interviewer: Which one is better?

Respondent: Having someone to talk to[…] not many people know the meaning of companionship (Female, 18)

“I prefer ‘having someone to talk to’ [than companionship]. Companionship just it doesn’t really explain what it is. And having someone to talk to is just having someone to talk to. It explains it without, I don’t know, it’s just better, clearer.” (Female, 18)

When asked to explain what they thought companionship meant, one interpretation is that it’s about “having someone to talk to”. However, there was also a belief that you can have someone to talk to without having the depth of relationship that might be implied by companionship:

“It is a massive thing because I suppose it is kind of separate from parents, I wouldn’t think of a parent as a companion. Having a companion is, it’s someone who you can talk to about anything. With parents there will always be certain topics that you just can’t talk about because it’s not the kind of thing that you talk about. And it’s having someone around your age, perhaps who go through similar situations, who perhaps also doesn’t really know the answer. Sometimes you don’t need an answer you just want to talk through something and say how you’re feeling and get it out and having that mutual trust with someone or even multiple people is a huge thing just to get things off your chest. I think if I didn’t have that I wouldn’t talk to anyone really. I wouldn’t talk to my parents about it.” (Male, 17)

Similarly, a distinction was also drawn between having someone to talk in a professional capacity versus having a companion one could talk to:

“Well it may be because having someone to talk and a companion in my opinion, I would go to like probably my boyfriend or something like that, and maybe sometimes my dad. But someone to talk to can be like a counsellor or something like that. I’ve been through that and they’ve been my ‘someone’ to talk to. But I wouldn’t ever describe them as a companion. I’d call a companion someone to talk to and a companion like. The companion and someone to talk to can be used together but I don’t think, like in that sense of the way, but not in the other side of it. Does that make sense? It makes sense in my brain but.” (Female, 20)

5.2 “How often do you feel left out?”

5.2.1 Understanding of the question

Respondents were able to answer this question using the response options offered without a problem. Similarly to the findings for children aged 10 to 15 years, feeling “left out” was understood by the majority of respondents as not being included in social situations, activities or discussions:

“You're asking how often am I feeling like I'm left out of situations with my friends, with my family, people maybe not telling me things or inviting me somewhere, those kinds of situations.” (Female, 18)

5.2.2 Alternative interpretations

Respondents were asked if they could suggest an alternative to “How often do you feel left out?”. This helped to clarify that feeling left out could be experienced in a number of ways including:

• others not giving you an opportunity to join in activities, reflecting an externally imposed sense of exclusion:

“How often do you feel that you are not given an opportunity to engage in a pastime, or how often do you feel you're not involved in an activity.” (Male, 16)

• feeling unable to approach or talk to people or on a different level with those around you, reflecting a sense of emotional exclusion from others:

“Do you feel isolated, do you feel you can’t approach people? Do you feel you can’t talk to people, communicate and all of that.” (Male, 17)

“I think it’s about, you’ve got like communication, if you’re not on the same level, if everyone’s talking or everyone’s got a dynamic which you’re not part of, even when you’re with them you feel left out. It’s like, in essence, third-wheeling really.” (Male, 18)

It was also suggested that there are many ways that people can be left out, making this open to a range of interpretations:

“I think it's like anything if you're going to interview people about loneliness, you're going to have to ask them have you felt left out. But isn't it left out in what sense? Kind of left out of what? So, you can see a way it can be an open-ended question”. (Female, 23)

There are a range of situations in which young adults said they felt left out:

“Like sort of, of a conversation or something like that. If you’re sat around a table or even if people make plans. Or, you know, if you feel like, I feel like things go on around me that I’m unaware of but I feel like I should be aware of. Like family, like group chats and stuff, there’s a lot of things that go on. And then they’ll, like my family will chat but they’ll chat separately. And then they’ll be like oh you knew this. No, I didn’t, you didn’t tell me. There’s a lot of, you know, that sort of thing. Or you’re sort of sat there at the table and you’re just like I don’t really feel like I have any contribution or no one’s bringing my opinion into it or. I would say if you went out with friends and had plans, but that doesn’t really, I don’t really go out or anything so.” (Female, 20)

With such a wide range of ways that individuals could feel left out, this may have implications as to how often it occurs and hence how frequently respondents report it in surveys. However, this specific question was generally answered quickly by respondents and only on reflection did they think more deeply about its full meaning.

There were also comments from disabled people suggesting that isolation can be closely related to social inclusion and accessibility, and a sense of exclusion may be a very common experience:

“Do you feel included in your community? Do you feel included in your friendship groups? Do you feel like you’re not involved in those[…] it’s quite personal in terms of access and disabled access there is a lot of social inclusion issues. So feeling left out is something I feel every single day.” (Female, 21)

5.3 “How often do you feel isolated from others?”

5.3.1 Understanding of the question

As with the previous two questions, respondents were able to answer the question on how often they feel isolated from others with ease. Their interpretations focused different aspects of isolation:

• socially or emotional isolation, implying little or no meaningful communication with others even though others may be physically close

• physical isolation through distance from others though contact may still be possible remotely (for example, by phone, text or social media)

These different dimensions of isolation may mean people take different things into account when responding:

“How often do I feel like I'm kept separated from other people. I can't maybe contact people or talk to them. Yeah, I'd say that was a little bit more of a challenging question for me to think about[…]. Yeah, I'd say because is it like a physical isolation from them or is it, yeah, I wasn’t too sure where to take that question[…] Do we mean that I can't contact somebody on my phone? Do I mean that they don’t want to talk to me? Do you mean I can't physically see them? Like, that is what I mean like I didn’t know quite where to take that question.” (Female, 18)

5.3.2 How often do you feel alone?

Respondents were also asked what they thought the difference was between “How often do you feel isolated” and “How often do you feel alone”? The questions were thought to ask very similar things, though some distinctions between them were noted. For example, being alone can be a positive choice, whereas isolation is unlikely to be. Similar findings emerged as for children and young people, in which “isolation” was seen as a more extreme state than “feeling alone”:

“It depends on the value of the question because if you ask people how often do you want to be alone, I want to be alone most days. I don't want to talk to people. But if you'd ask somebody how often do they want to be isolated, they'd say they wouldn't want to be isolated because it sounds a bit more negative and dire in a sense than how often do you want to be alone. Because you can be in a busy place where it's noisy and people are talking – I want to be alone. I can't be doing with this right now. I just want a cup of coffee. And you could have a grumpy day, like I don't like mornings; I just want to be alone until 10:00am. But you wouldn't say I want to be isolated till 10:00am.” (Female, 23)

“How often you feel alone is not really a big thing. It's like because everyone feels alone at some point. Literally everybody will feel alone at some point in their life. I feel like with isolated not as many people will feel that, […] and people will feel like they won't really have much support.” (Male, 16)

Isolation was also thought to reflect a more prolonged and negative mental state than feeling alone, which could be a transient experience:

“For me kind of isolation is more kind of impacting than being left out. If you’re left out you can kind of renew it. You can try again, go to a different place with your friends or do something else. You can fight to change being left out. Whereas feeling isolated is something very internal and it’s an emotional symptom of being left out for so long that you kind of internalise. So for me kind of feeling isolated is now a mental barrier where I can’t let people in emotionally or I feel like I can’t do certain things or I won’t do certain things because of my previous experiences.” (Female, 21)

These feelings were echoed by other respondents who also felt that being alone could be a first stage towards being isolated. This point was reflected in an example from a young person who described a transition from “feeling alone” to “feeling isolated”, which happened after increasingly withdrawing from situations in which they might feel left out. Self-imposed isolation was also something noted by respondents as potentially associated with mental health issues.

5.3.3 Preferred wording

Respondents preferred “feeling alone” to “isolation” because it was clearer wording, but they understood the meaning of isolation and applied it appropriately in the examples they gave. It was not associated with punishment as it was with those aged 10 to 15 years.

5.4 “How often do you feel lonely?”

5.4.1 Understanding of the question

Respondents were also asked the direct question, “How often do you feel lonely?”. They did not find this difficult to answer, but noted it felt similar to the other questions:

“I'd say it was quite similar to the isolated one. I'd look at it and feel like how often do I feel like I'm by myself, alone, don’t have, like I said, people to talk to, people to kind of offload any of my issues onto. And yeah just feeling very like you can't talk to people, trust people, yeah, in those situations.” (Female, 18)

“Well it’s just a combination of all three above together. It’s like for me it just feels like a summary.” (Male, 23)

5.4.2 Alternative interpretations

In keeping with the findings from younger respondents, “being lonely” was differentiated from being alone:

“You can feel lonely in a room full of people[…] it’s not a physical thing. Lonely is a state of mind.” (Female, 17).

Respondents clearly associated alone with a physical attribution, using the term “being alone”, and lonely with an emotional one using the term “feeling lonely”:

“Loneliness which is where you don’t have anyone to fall back on, where you have that kind of emotional impact of having to go at it on your own, whether that be physical or emotional[…] I think being alone is a very physical thing if you don’t see anyone. Being lonely is the emotional thing of not feeling understood, not feeling valued, not feeling like you have anyone to lean on, like you have no one who respects you and respects your dignity.” (Female, 21)

During the cognitive interviews, respondents were asked how they felt about the response options for the questions and whether they would prefer specific alternatives.

6.1 Children and young people’s use of the response scales

6.1.1 Alternative response scales considered

A specific suggestion put to children and young people was whether they would prefer to answer the questions on a 0 to 10 scale with 0 being “Never” and 10 being “Always” or using labelled response categories such as: “Hardly ever or never”, “Some of the time” or “Often”. The labelled response categories were used as part of the question testing so they had experience of this.

Labelled response scales

Those who preferred labelled response categories felt that words were easier to understand and more meaningful in terms of loneliness than numerical categories:

“I think words are better[…] it lets me draw more on my thoughts because it’s prompting me more than a number I guess.” (Male, 14)

“[…] it’s easier to understand if you say words. People might not understand if you use a scale.” (Female, 14)

Numerical response scales

Among those who reported that they would prefer the 0 to 10 scale, this was primarily because this approach enabled more response options:

“I think I'd prefer a 0 to 10 scale just because it's got more of range that you could like go into a bit more detail.” (Female, 13)

“[…]You’ve got more of a range instead of just three answers[…] I think maybe numbers it’ll have a wider range of choice.” (Male, 13)

A possible trade-off was also noted between more choice in a 0 to 10 scale versus ease of understanding in response options with labels:

“Because there’s more choice. It’s easier to give a better answer[…] [but] it’s easier to understand if you say words. People might not understand if you use a scale.” (Female, 14)

A final observation is that children who answered “hardly ever or never” tended to treat this as two distinct response options and pick one:

Interviewer: “How often do you feel that you have no one to talk to: hardly ever or never, some of the time or often?”

Respondent: “Never.”

6.2 Young adults’ use of response scales

6.2.1 Alternative response scales considered

As with children and young people, young adults (aged 16 to 24 years) were also asked their views on whether they preferred the three-item labelled response categories used when they answered the questions (“Hardly ever or never”, "Some of the time" and "Often”) or a 0 to 10 scale with 0 being never and 10 being always.

There were advocates for both the labelled response scale and the numerical scale with spontaneous suggestions for other possible approaches including: a 0 to 5 numerical scale; a labelled five-item scale; and an open text box to enable people to give more details regarding how they feel.

Fans of the three-item labelled scale felt it offered sufficient response options. Similar to the findings for children and young people, there was a view it was easier to answer in words than numbers:

“Words is more straightforward[…][and] lets you be a little bit more specific[…] probably easier.” (Male, 19)

Those who preferred labelled responses thought it was difficult to quantify an emotion:

“I think that because it's sort of like a subjective thing and a sort of emotional topic I don't think you can like quantify it and say 6 out of 10 because like your feelings can be different about different matters[…] I think the other answers are probably better for like description purposes because even if you said like, say if I felt left out eight of ten but because being left out can be different circumstances so I think it's more difficult, I don't think it's like appropriate to quantify it.” (Female, 24)

The use of words in the response scale could also help to anchor the question in people’s own experiences:

“I quite like the hardly ever, sometimes answer for this[…] because it helps you to think of specific situations and it helps you to clarify how often you feel that emotion and how often it kind of comes up[…] because I think it is one where you have to think about literal events or times that you felt like that.” (Female, 24)

By contrast, the main advantage cited with the numerical scale was the wider breadth of response options:

I thought they were quite good categories. I mean I could choose. I'm trying to think. Maybe it would be nice to have a larger scale made like a 1 to 10 kind of thing, rather than limiting that. If you said one is never, and I suppose it's giving you quite a range then, but maybe that would be nicer to do it that way.” (Female, 18)

“With three categories, it was three that you gave us before? I just think there’s not much like what’s the word, I can’t think of the word – leeway. Whereas if it’s like one to 10 you can proper put a finger on it”. (Female, 17)

Young adults were clearly very used to taking part in surveys and this was also reflected in their responses:

“I think if you had a scale like you do like online ones, you can have all the time, some of the time, occasionally, hardly ever and then never, like a scale of five or so, and you give a little bit more differentiation there's more places people can fall[…] But it might be easier for people to relate to certain things: you might get more accurate or direct answers[…] I wanted to say a mix between all of the time and some of the time for some of those questions and occasionally and never, but if there wasn’t an in-between option.” (Male, 16)

Those who preferred more response options felt this would allow a more specific response, better reflecting individual experiences:

“I think that kind of gives it a bit more of a sliding scale because sometimes it's, you sit sometimes you think hardly never and sometimes it's finding that spot to explain your answer.” (Female, 24)

“More of a scale might have been better[…] I just think it’s easier to gauge because you can sort of giving something out of ten is I think more specific. Because sometimes you might have put yourself between a category or someone might find it hard to think between sometimes and often or something like that.” (Male, 16)

Respondents were also asked about the environment in which they would prefer to answer these questions. We were particularly interested in exploring issues of where respondents would be most comfortable answering the question, confidentiality and the importance of support being available, if required.

7.1 Children’s and young people’s preferred environment for completing the questions

We asked children and young people whether they would prefer to answer the loneliness questions at home or at school.

7.1.1 At home

Among those who preferred to answer the questions at home, reasons given related to concerns about confidentiality and risk of embarrassment if peers or friends found out their responses:

“[At home] there’s nobody to, there’s nobody watching you really because you can trust your mum and not [others] at school[…] because usually they would go around telling everyone. That wouldn’t really be that comfortable.” (Male, 12)

“Probably at home, because I don’t really want to be putting the same answers down as my friends or talking about it with them afterwards.” (Female, 14)

“[…] what happened with us after the height and weight checks everyone was going ‘oh what’s your height?’ ‘oh yes, I’m taller than you’. Like not yes like ha ha, I’m better, like yes, I’ve grown. Like that. So, people might feel under pressure to tell other people. So, it would be better if you do it at home and then no one knows when you’ve done it and they can’t ask.” (Male, 11)

Creating a “safe” environment in which children and young people can answer the questions is important to enable them to answer honestly and to be clear there won’t be negative repercussions from doing so.

Completing the survey on a computer at school was viewed as problematic in this regard if others could see their screen:

“It might be better at home because at school lots of people look at each other’s screens. And go oh why are you looking at that. If you give me something I won’t tell that you were looking at that. Or they can find out information you don’t want them to know.” (Male, 11)

The importance of not having to answer in front of others who aren’t trusted completely was also highlighted by respondents:

“Because at school there’s teachers there and you could feel like your answer could just like, if they didn’t expect you to answer that you could just feel like weird or judged.” (Female, 14)

“[…]So, with me, if I had to answer in front of my parent, I don’t think my answers would change because I trust my parents. But maybe if someone didn’t trust their parents as much, or if it was in front of a teacher or someone like that, they might be inclined to lie.” (Female, 13)

7.1.2 Preference to answer loneliness-related questions alone

Concerns about confidentiality and fear for potential embarrassment or stigma, were also noted as reasons for preferring to be alone when answering loneliness-related questions:

“If family or friends are there they probably feel like they want to put the same answer as their friends, or if something was on their minds, they wouldn’t want their family to know[…]" (Female, 14)

Although there were concerns expressed about being able to see others’ screens at school, a self-administered survey, by computer, was suggested as preferable to an interviewer-led approach:

“I think they'd prefer to do it on a computer because I feel like some people, if someone's an introvert or something like that, they'd feel more comfortable doing it [by] themselves […]. They might feel inclined to be less honest if they're in front of people. […] It might make them inclined to lie.” (Female, 13)

7.1.3 At school

Among those saying they would prefer to answer the questions at school rather than at home, they were still keen to suggest steps that should be implemented to ensure confidentiality or privacy and avoid others knowing their answers. These suggestions largely related to answering the questions alone:

“A private room at school with a laptop where you can just submit your answer and it’s gone. And you can choose whether people know your name or they don’t.” (Male, 11)

“Probably by just letting me get on by myself probably because it would be a lot easier I’ll find.” (Male, 12)

7.2 Young adults’ preferred environment for completing the questions

Similarly, to children and young people, young adults (aged 16 to 24 years) were also asked where and how they would prefer to answer the questions, focusing particularly on home or school or college.

As with the younger respondents, a consistent theme was that it was important to have privacy when answering the questions. Without this, people may not be inclined to answer openly:

“When you’re with your group you act up in front of them don’t you? Act like the big one. So they’ll not answer honestly.” (Female, 18)

“Not comfortable. I don’t think I’d give a true, especially if I was around my parents, I wouldn’t want, if it was people I didn’t know that wouldn’t bother me, but people I’m close to I wouldn’t want them to know.” (Female, 20).

7.2.1 The right conditions for answering the questions

Another similar idea to the findings for children and young people is that it is important to provide a safe and private space for answering the questions, where they feel at ease and can answer at their own pace:

“[…]it would be at home because you'd have the time, you'd have the facilities. You wouldn’t have to rush. You'd make yourself a cup of tea and yeah, I've got to do this now. And it'd be just out of convenience really.” (Female, 23)

“And a lot of times in college, I know when we got surveyed and stuff you never really finish. You just want to get it done as quickly as possible and get rid of it. That's usually what it was like at school because that’s just the way it is. But at home, I think you’d answer it a bit more in-depth and stuff, yeah.” (Male, 20)

“Home’s probably better[…] first year of uni, it’s a little bit more, you don’t quite have your own space, but at home you definitely do.” (Male, 18)

Place was also an important consideration for people with additional needs, with home noted as potentially being an easier and more comfortable option:

“Personally, I’d prefer to do it in my own space because it’s a lot less clinical. And I know a lot of people, especially with additional needs know that there’s a lot of anxiety around going to a different place already with oh is it going to be accessible, are the people’s attitudes going to be accessible? Whereas when you’re in your own environment it’s a lot kind of easier to relax into the questions rather than having to go somewhere.” (Female, 21)

7.2.2. Influence of people around you

Many respondents thought that it could be problematic if they had to answer these questions in front of others including friends, family and carers. There are clear implications for ethics and data quality if respondents are asked to complete surveys in front of other people where the confidentiality of their answers cannot be guaranteed:

“You probably wouldn’t answer it clearly because they wouldn’t want to tell their family and friends that that might be a problem. If I had my mum and dad behind me, if I did have a lack of companionship, I wouldn’t answer that in front of my family. Probably the reason being that you wouldn’t want to admit it, maybe, if you were a young adult or a young child or whatever.” (Male, 20)

“Some people try and impress people and they want to try and impress their friends and say oh no I'm never left out. People just wouldn’t answer that.” (Male, 20)

These issues should also be considered carefully and sensitively in circumstances where people may need support from a carer to participate:

“I have only got capacity to go somewhere else when there’s a person with me. And quite frankly going somewhere else I’d have to watch who I then said it to.” (Male, 16)

7.2.3 Potential to get support

A further view was that answering these questions around others might promote more open discussion and greater support:

“Probably school[…] if you had friends around you and you answered the question in front of them, they might realise.” (Male, 19)

In terms of accessing support, young adults differed in their view as to where this might be more easily available at home or school and depended on the type of support required:

“Probably on a computer at home, or at school. Having it at school[…] because you’ve got the teachers and your schools friends around there. So they help you understand the meaning of it. I don’t understand things or read well[…] Having your family to talk to about it. And probably being too scared to if they don’t want to talk about it.” (Female, 17)

“I think it would be better to do it at home because you can just talk about more. I don’t know, if you’re at school you’re like around other people and you wouldn’t feel like you could talk about it as much[…] I just think you could just tell the truth at home.” (Female, 17)

7.2.4 Mode of response

Respondents discussed the advantages and disadvantages of a range of modes of completion, including face-to-face interviews, telephone and online. A computer-based interview was appealing for a number of reasons including: ease, speed of completion and could be completed at home in private:

“[…]it'd probably be a bit easier to click an option rather than just actually admit it over the phone probably, yeah[…] on your own, probably at home. If you were say on your phone or on your laptop or something it would probably be the best way to answer it, yeah.” (Male, 20)

Some concern was expressed about telephone interviews, which could be less comfortable for respondents:

“I definitely wouldn’t do the phone one[…] I get very nervous. I don’t mind face-to-face because you can get your message across to someone properly[…] I don’t mind the internet because it’s quick and easy. I’d probably pick the internet because I can easily slot that into my time.” (Female, 17)

There was also concern about including the questions on a household survey, both because of a possible lack of privacy and the potential for raising difficult issues within the family:

“Yeah. If there was a way that it could be, you sit somewhere, do it and it’s gone that’s it. Done, dusted, they won’t see it. Whereas if it’s sent out to a household they could, I know what my mother’s like.” (Female, 20)

Finally, despite the privacy potentially afforded by completing the questions online, there were also concerns raised about the security of internet-based modes of completion:

“With online, because a lot of big things nowadays is big scares like something's going to be hacked or there's been a leak on iCloud or whatever, and I know that obviously won't affect a lot of things, but it would be quite scary in a way […]I'd probably say paper or, if the option's there, in person is quite nice. Because it's just like quite genuine and you can see someone's expression and see how they're reacting to this. But then with paper as well you write it down, it gets sent off[…]” (Male, 16)

7.3 Perceived impacts of the questions

7.3.1 Children and young people’s views and experiences

Children were asked how they felt when answering the questions and whether they thought the questions could be upsetting for others. Although they acknowledged that all the questions had the potential to be upsetting to those affected by the issues, they did not relate this to themselves – even those who said they had experienced aspects of loneliness or had no one to talk to:

“It could be upsetting for some people. If someone is going through a time where they are lonely and they’ve got no one, it could upset them […] I think they could feel quite embarrassed, because some people might find it embarrassing because they’re not as popular as anyone else.” (Female, 11)

They emphasised that the questions might be upsetting depending on the experiences of the person answering them:

"It depends what they’ve been through, because if maybe they were a really lonely person it might be quite upsetting for them, but maybe if, a person who’s had a lot of friends or maybe been a bit lonely it might be hard to answer it but you would know what to say in the end like I did. Or if you’re just a person who’s had friends from the beginning you probably would find it quite easy to talk about.” (Male, 13)

Respondents also noted that if people had experienced loneliness, this may discourage them from answering the questions honestly:

“Some people who do feel lonely might not want to admit it because it might not be like a thing they want to admit, because like they might feel like it makes them sound weak or whatever or like they don’t have friends or whatever.” (Female, 14)

“It could be unnerving for the people who are lonely because they might not want to say and they could be like yeah I’m never alone when you can see in their eyes that they are.” (Male, 11)

Although they felt the questions could be upsetting, respondents also recognised the value of asking about loneliness and encouraging people to talk about it:

“I think it's quite important to keep asking them because most children wouldn’t reach out and say I feel left out of this. Whereas, if you're asking them, it's giving them an opportunity to share how they feel. […] Because I think that would be one of the main reasons that people could start to become isolated and start pushing people away if they think if I'm constantly being left out, why should I make efforts with people, why should I go and speak to someone. And I think that could be the root cause of leading to like things becoming worse.” (Female, 13)

7.3.2 Young adults’ views and experiences of answering loneliness questions

Among the young adult group (aged 16 to 24 years), as with the younger group, respondents felt the questions could be upsetting for those who were lonely or had little social support:

“Yeah, I know there would be [people who would be upset]. Like a couple of my friends, like people at my school, people I know, people I care about, I know they'd find it quite hard to answer that question due to upbringing or the situations they're in at the moment. And yeah, I think that’s just it. Like they would find it quite difficult to answer due to circumstances.” (Male, 16)

“Quite upsetting if that’s how they felt. Like if they were alone and they got asked on it, it would be a bit awkward[…] they might just reflect on themselves that they’ve got no one to talk to.” (Female, 17)

Asking the questions and moving on quickly to other topics was suggested as a way to make it easier for people to answer and to minimise potential upset:

“Not as much, because it’s just one little question. If you’re having a conversation about it you’ve got to talk about it.” (Female, 17)

“My feeling is just answer it, move on.” (Male, 16)

However, those who said that they did feel aspects of loneliness also said that the questions were not particularly upsetting to them:

“I felt OK. I'm relatively open about talking. I talk quite a bit. But yeah, I feel if I don’t talk about it then what's the point in having the ability to speak. Like I've got to talk about something and if a question's asked, I might as well answer it. And I don’t really feel uncomfortable. I was all right with it, to be honest, pretty good.” (Male, 16)

As noted previously, due to the sensitivity of the topic, providing a safe space where people can give honest answers in privacy is particularly important to avoiding adverse impacts:

“Maybe they wouldn’t want to admit it, I think. It's quite a sensitive topic, really, to admit that you're lonely. It's quite a tough thing to do but I feel like if you were with someone, one-to-one you would answer that quite honestly.” (Male, 20)

“I can see how it could be [upsetting] if it was asked to a certain type of person. And I mean, obviously there's people in school, you see it going on, they can be bullied. They'd be laughed at. There were a few people in my school that ended up moving schools. You can see how it probably would upset some of them. […] I don't know. Only people that are probably lonely.” (Female, 23)

8.1 Findings and recommendations on question wording for children and young adults

Children and young people aged under 16 years understood the questions well and were confident about how to respond to the adapted, plain English versions tested.

There were a range of interpretations given to the word “companionship” among children and young people as well as for those aged 16 to 24 years. This was not an easy term for people aged under 25 years and there was a lack of clarity about the nature and depth of relationship implied by it. Apart from varying interpretations from platonic to romantic relationships and with different degrees of closeness, a companion might also include a pet, and for those requiring assistance with independent living, the term is also connected to professional carers (for example, companion care).

Much of the ambiguity was removed for children aged 10 to 15 years by adapting the question from “lacking companionship” to “having no one you can talk to”.

Despite the varying interpretations of the question among those aged 16 to 24 years, we do not recommend alternative wording of the “companionship” question for adults. This is because it is part of a well-tested and validated scale currently in use on major surveys of adults in the UK and internationally, and it is important to retain the question wording to maintain comparability. For those undertaking surveys where these different interpretations of the question may be particularly relevant (for example, surveys of those with long-term health conditions or disabilities), it may be helpful to offer further clarification of what is meant by “companionship”.

For children and young people under 16 years, the word “isolated” could be viewed as an externally-imposed separation from others, associated with punishment for bad behaviour at school. For both those under 16 years and those aged 16 to 24 years, the term was thought to imply a prolonged, severe and deeply-felt separation from others. For those aged 16 years and over, there was more recognition that isolation may be externally imposed or self-imposed and may involve a sense of emotional separation from others, physical separation or both.

For those under 16 years, we adapted the question about feeling “isolated” from others to ask, “How often do you feel alone?”. Our testing of this showed this was well understood by children and was not associated with the idea of punishment. Different interpretations were noted in relation to “being alone”, which could be a positive experience (for example, taking time out for oneself) and “feeling alone”, which implied a more negatively experienced sense of aloneness not of one’s choosing.

The question, “How often do you feel left out?” was well-understood by children and young people aged 10 to 15 years and those in the older age group aged 16 to 24 years. For children and young people, this was interpreted as being excluded from activities or friendship groups and might be temporary or more enduring. Some in their mid-teens and beyond also noted that this could relate to wider exclusion from full participation in society, such as social exclusion that may be experienced by minority groups. This was specifically noted by respondents both in relation to ethnicity and disability.

The direct question on loneliness, “How often do you feel lonely?” was well understood by respondents of all ages and was viewed similarly to the question on frequency of “feeling alone” for children and young people and to the question on frequency of “feeling isolated” for young adults. For both children and young adults, the term “lonely” was clearly associated with a state of mind and it was widely acknowledged that one could feel lonely even in the company of others.

8.2 Findings and recommendations on response scales

All the questions were tested qualitatively using a three-item response scale. This is consistent with how the questions are currently asked on the English Longitudinal Study of Ageing (ELSA) and Understanding Society.

When asked about ease of using the response scales and other possible options, a preference for more response options was expressed as the three-item scale didn’t allow much distinction for expressing how frequently people feel a particular way. Coupled with this, there was a concern that the three-item response scale may not allow sufficient sensitivity for monitoring changes in the prevalence of loneliness over time.

To address these issues, we have recommended that the three-item response scale be retained for the first three questions based on the UCLA loneliness scale while a five-item response scale is used for the final, direct question on loneliness. This has the advantage of maintaining consistency with ELSA and Understanding Society on the first three questions from the UCLA scale, while ensuring that the final question is consistent with the loneliness question on the Community Life Survey (CLS).

One disadvantage of this solution is that the response scale used for CLS includes the word “occasionally”, which experts from our Technical Advisory Group suggested may be difficult for some children to read and understand. This was not included in the main cognitive testing, as it only arose in relation to the perceived limitations of the three-item scale highlighted during the testing itself.

As part of scoping work on previously used loneliness measures, we have found that a similar question on loneliness with a five-item response scale was used for several years among 10- to 15-year-olds on the British Household Panel Survey (predecessor to the Understanding Society study). That scale included the word “occasionally” and we have not found any evidence to suggest it was problematic for respondents. To understand any possible issues with this more fully, we are working with The Children’s Society to test this response scale among an additional group of children and we will update our guidance with any further suggestions that may be advised to maximise children’s comprehension of the response options.

Although more plain English versions of the scale exist, which could be used instead, it would be very helpful to have a single question on loneliness asked in the same way to everyone from the age of 10 years upwards and comparable with the existing prevalence measure of loneliness from the CLS. Our recommendations reflect this goal.

8.3 Preferences and recommendations on how and where to ask the questions

On the advice of The Children’s Society, we also asked children and young people where and how they would prefer to answer these questions. This arises from a concern to ensure that support is available if they are in any way upset by the questions or would like to discuss them further.

Our conversations with children and young people reflected an awareness of the possible need for further support among those particularly affected by the issues raised, but along with this was a strongly-felt suggestion that ensuring privacy and confidentiality to respondents should be an important priority. This was viewed as important regardless of the interview setting, but young people particularly worried about peer pressure at school to share responses. They also worried about their responses being seen by others, resulting in teasing or bullying at school or difficult conversations with family members if completed at home. Fear and embarrassment associated with others knowing their responses could result in a lack of honesty in how young people respond to the questions. This, along with possible embarrassment about answering the questions face-to-face or over the phone, was also associated with a preference for self-completion formats, either online or on paper, and administered in a way that guarantees confidentiality.

This is possibly reinforced by our experience with the cognitive testing, which involved asking these questions early in a face-to-face interview that usually took place in the respondent’s home. Their responses to the initial questions often suggested that they were not often lonely. Later in the interviews when they were possibly more at ease, they sometimes gave clear examples of feeling lonely, being left out or having no one to talk to. They also gave examples of other people whom they thought were lonely.

It is also important to remember that these questions will not normally be asked in the context of a survey focused entirely on loneliness. Generally, surveys cover many topics, with loneliness only expected to comprise between one and four questions and taking no more than two minutes of total survey time. In this context, confidentiality is still important, but loneliness may not be the main issue young people want to discuss (or want to avoid discussing) afterwards. As part of a longer and wider-ranging survey, young people may be more comfortable answering the questions.