A Government Statistical Service perspective on official estimates of calorie consumption

The Behavioural Insight Team’s ‘Counting Calories’ report explains the apparent decrease in calorie intake over time by presenting evidence that calorie consumption is underestimated and also suggests that the degree of underestimation has increased over time. This article provides a GSS response to that report.

Cyswllt:
Jen Woolford

Last revised:
8 August 2016

Cynnwys

  1. Collaboration
  2. Summary
  3. Background
  4. Implications of under-reporting in the LCF for National Accounts and other official statistics
  5. Plans for future improvements
  6. Annex A: Overview of dietary data collection methods and derivation of nutrient intakes from the National Diet and Nutrition Survey
  7. Annex B: Doubly labelled water assessment of under-reporting in NDNS (NDNS year 1 to 4 report)
  8. Annex C: Overview of data collection methods and the derivation of nutrient intake from the Living Costs & Food Survey

Argraffu'r Methodoleg

Lawrlwytho fel PDF

1. Collaboration

The article was produced in collaboration with the Department for Environment, Food & Rural Affairs; Public Health England; and the Department of Health.

Government Statistical Service (GSS) logo

Nôl i'r tabl cynnwys

2. Summary

Understanding calorie intake is central to developing policies to tackle obesity. While official statistics on diet and nutrition are based on best practice data collection methods, it has long been recognised that under-reporting in dietary surveys means that official estimates of calorie consumption are likely to be underestimated. The Behavioural Insight Team’s “Counting Calories ” report provides an analysis of official estimates of calorie intake in the context of evidence around energy expenditure and obesity levels. The report explains the apparent decrease in calorie intake over time by presenting evidence that calorie consumption is underestimated, and also suggests that the degree of underestimation has increased over time.

The approach to collecting data on food purchases and consumption is reviewed regularly, as are the methods used to translate this information into estimates of energy (calorie) and nutrient intake. New technologies such as web-scraping or using smart phones to record diet have the potential to deliver significant improvements. These are being considered but are not ready to be introduced in the near future.

However, there are a range of data sources that could help improve our understanding of calorie intake. These include:

  • store scanner data

  • supermarket product information (which could be obtained via web scraping) linked to electronic receipts

  • information on food wastage

  • studies using doubly labelled water or other biomarkers of energy and nutrient intake

Bringing these and other sources together would help our understanding of under-reporting and could improve the quality of official statistics. A cross-department team is being established to analyse these sources of information and how they relate to each other. We will publish plans for this work shortly.

Nôl i'r tabl cynnwys

3. Background

Obesity is estimated to affect around 1 in 4 adults and 1 in 5 reception-aged children in the UK. It is an important policy concern and the government childhood obesity strategy is due to be published shortly. A particular challenge in tackling obesity is understanding people’s dietary habits and how these are changing.

National Statistics on diet and nutrition are based on survey data: the National Diet and Nutrition Survey (NDNS) and the Living Costs and Food Survey (LCF) (see annexes for more detail). Both surveys adopt recognised best practice data collection methods.

Official Statistics on calorie consumption derived from the LCF are based on household purchases and published by the Department for Environment, Food and Rural Affairs (DEFRA), who take food and drink expenditure data and apply various factors for nutrient content and consumption rates (see Annex C). These data show a long-term downward trend in calorie consumption. Over the period 2000 to 2001 up until 2011 to 2012 there was a decrease of approximately 6.8% in the energy content of food purchases.

Official Statistics on calorie intake estimated from the NDNS, which collects data on food consumption at the individual level, show a similar reduction: a 5 to 6% decline over the same period, comparing reported intake in 2000 to 2001 with 2008 to 2012. NDNS is a smaller sample size than LCF but it focuses on food consumed by individuals and collects more detail on the types of foods consumed (see Annex A).

Under-reporting of food consumption has long been recognised in dietary surveys around the world, including those in the UK. Both the LCF and NDNS acknowledge that under-reporting exists in their surveys. Under-reporting may result from a number or combination of behaviours, for example omitting to record foods or drinks consumed, whether intentionally or otherwise, underestimating quantities consumed, or changing usual consumption as a result of being asked to record diet. Changing eating patterns towards more eating out and snacking may mean that survey participants tend to under-report to a greater extent now than they did in the past, although the evidence around this is limited.

Public Health England (PHE) have investigated ways to tackle under-reporting in dietary surveys and concluded that – while under-reporting can be mitigated by using the best dietary assessment methods, delivered by trained fieldworkers – it is not possible to eliminate it entirely. Furthermore, there are no robust methods currently available to correct or adjust datasets for under-reporting. Despite this, these surveys provide valuable data to assess nutrient intakes, although the existence of under-reporting means that care is needed in the detailed interpretation of data.

The latest NDNS report, published in May 2014, reports the results of a study to assess energy expenditure in a small sub-sample of NDNS participants using a technique known as doubly labelled water (DLW). Results from the DLW study suggest that reported energy intake in adults was on average 34% lower than energy expenditure measured by DLW, although there was a wide variation between individuals. This gives a clear indication that there is substantial under-reporting in NDNS (see Annex B). However the small size and lack of representativeness of the DLW sub-sample means that these estimates cannot be used to adjust overall estimates of calorie consumption reliably.

The Behavioural Insights Team report provides a useful analysis of official estimates of calorie consumption derived from these surveys against other sources of data, and we welcome the additional insight this analysis provides.

Nôl i'r tabl cynnwys

4. Implications of under-reporting in the LCF for National Accounts and other official statistics

Expenditure data from the LCF is used in a range of National and Official Statistics. For these statistics it is possible to take account of the under-reporting to minimise any impact:

  • National Accounts make an allowance for under-reporting in the LCF by adjusting against other estimates, for example, retail sales of food and in the balancing of the separate measures of income, output and expenditure

  • estimates of alcohol and tobacco in household final consumption expenditure (HHFCE) in the National Accounts are derived from administrative data from HM Revenue and Customs rather than the LCF as it is recognised that estimates in household surveys of this consumption are biased

  • the effects of taxes and benefits on household income (ETB) analysis, based on the LCF, contain estimates of indirect taxes. These taxes are calculated from expenditures recorded in the LCF on products such as alcoholic drinks, tobacco, ice cream, soft drinks and confectionery. To account for the known under-recording of these items they are grossed up to National Accounts aggregates. This makes the assumption that true expenditure in each case is proportional to recorded expenditure; however, where possible, these estimates are also benchmarked to administrative data from HM Revenue and Customs

  • Consumer Price Index (CPI) weights are consistent with HHFCE

We are therefore confident that the quality of these official statistics is not affected by any under-reporting in the LCF.

Nôl i'r tabl cynnwys

5. Plans for future improvements

The methodology of all GSS Surveys is reviewed regularly. The Living Costs and Food Survey has recently been subject to a National Statistics quality review (NSQR), which is due to be published shortly. The collection of data on food and non-alcoholic drink within the diary was reviewed, although no immediate diary design related cause for under-reporting was identified. However, the NSQR recommends carrying out further analysis to understand more about under-reporting, as well as the need to update the layout and content of the current paper diary. There are also recommendations to improve data collection methods by making use of new technology and commercial data sources. For example, it may be possible to semi-automate coding of purchase information from scanned supermarket receipts and deliver improvements by gaining access to store scanner data. We are currently preparing a response to the NSQR report, outlining how these recommendations will be taken forward.

For NDNS, the main focus has been on reducing under-reporting as much as possible by ensuring that interviewers are highly trained in techniques to obtain complete and accurate information from participants and making best use of the contact time they have with participants to ensure that the food diary is the best possible record of what was actually consumed.

Expert workshops have been used to review the methods used in the NDNS. The most recent of these concluded that current methods should be retained in the short term, but that the potential for use of technologies such as digital cameras or smart phones and automated coding or processing systems should be considered. This resulted in a review of the best use of new technologies in the NDNS, conducted in 2014. This concluded that several new technologies to assess diet showed promise but would require further development, validation and assessment of feasibility before being suitable for use in national surveys. Public Health England (PHE) keep these developments under review but have no immediate plans to introduce new technologies.

However, there are a range of data sources that could help improve our understanding of calorie intake. These include:

  • store scanner data

  • supermarket product information (which could be obtained via web scraping) linked to electronic receipts

  • information on food wastage

  • studies using doubly labelled water or other biomarkers of energy and nutrient intake (see Annex B)

Bringing these and other sources together would help our understanding of under-reporting and could improve the quality of official statistics. A cross-department team is being established to analyse these sources of information and how they relate to each other. This will include consideration of the outcomes from the LCF NSQR, such as the use of new technology and commercial data sources. We will publish plans for this work shortly.

Nôl i'r tabl cynnwys

.Annex A: Overview of dietary data collection methods and derivation of nutrient intakes from the National Diet and Nutrition Survey

NDNS collects data on food consumption in individuals using a four-day diary. Survey participants are asked to record everything they ate and drank for four consecutive days, both inside and outside the home. Interviewers instruct each participant on how to complete the diary and carry out a food diary check visit on the second or third day of recording, either in person or over the telephone. The check aims to collect missing detail for foods recorded, improving recording for the remaining days and also providing encouragement to participants to continue recording. Interviewers collect the diary and check the remaining days no later than three days after the final day of recording.

Portion sizes are estimated using household measures or weights from labels. Adults can also describe their portion sizes using photographs of ten frequently consumed foods provided in the diary. For children, an age-appropriate version of a food photograph atlas is used during the diary review process. The atlas presents a range of portion sizes for 44 commonly consumed foods for which portion size estimation is difficult. For children aged 11 years or under the food diary is kept by a parent with the child and/or carers contributing information as appropriate. The foods and drinks recorded in the diaries are coded to a coding frame based on around 4000 food codes. Each food code is associated with around 50 nutrients. Nutrient data is drawn from PHE’s analytical programme, supplemented by manufacturer’s information.

We know that under-reporting of food consumption exists in the NDNS as in dietary surveys around the world. There are considerable challenges in collecting robust dietary intake data and under-reporting has been recognised for some time in dietary surveys. Under-reporting may result from a number or combination of behaviours, for example omitting to record foods or drinks consumed, whether intentionally or otherwise, underestimating quantities consumed, or changing usual consumption as a result of being asked to record diet. Changing eating patterns towards more eating out and snacking may mean that survey participants tend to underreport to a greater extent now than they did in the past, although the evidence around this is limited. Public Health England work closely with the NDNS contractor team, led by scientists at the MRC Centre for Human Nutrition Research, to ensure the methods are the best available and the data collected are robust and provide a sound basis for PHE’s work.

NDNS data collection methods

The data collection methods used in the NDNS are regularly reviewed and new improved methods sought. Although under-reporting may not be an explicit driver in each case, it remains a factor in all considerations of methods.

NDNS uses the most robust and accurate methods available to collect dietary data and the data collection methodology is designed with the aim of capturing as complete and accurate a picture as possible of what people actually ate and minimising misreporting. Features of the methodology include:

  • use of professional interviewers trained in maximising response rather than dietitians or nutritionists

  • intensive interviewer training on the dietary data collection methodology focussing on the importance of getting complete, accurate and detailed information from participants. This includes prompts to check for missing information such as lack of drinks and snacks

  • participants provided with user- friendly instructions on how to complete the diary and the level of detail to provide. This information supplements face to face instruction from the interviewer

  • an interviewer visit or phone call mid-way through the diary period is built into the protocol specifically for the purpose of checking completion of the diary, filling in missing detail with the participant, answering their questions and encouraging them to continue

  • time built into the protocol for the interviewer to review the diary with the participant at the pick-up visit in order to fill in missing information and detail

PHE work closely with the contractor team to keep the survey methods under review and seek improvements. For example, the use of new technologies is being considered to make it easier for people to record what they eat1. NDNS data is widely used both in the UK and internationally and is subject to close scrutiny.

Notes for Annex A:

  1. Making the best use of new technologies in the National Diet and Nutrition Survey: a review; University of Leeds, March 2015
Nôl i'r tabl cynnwys

.Annex B: Doubly labelled water assessment of under-reporting in NDNS (NDNS year 1 to 4 report)

The latest NDNS report, published in May 2014, reports the results of a study to assess energy expenditure in a sub-sample of NDNS participants using a technique known as doubly labelled water (DLW). The DLW technique involves drinking a known dose of water labelled with two naturally occurring stable isotopes of hydrogen and oxygen. Urine samples collected over the following days are analysed to measure how quickly the isotopes are excreted from the body and from this energy expenditure can be calculated. Results from the DLW study showed that reported energy intake in adults was on average 34% lower than energy expenditure measured by DLW. Slightly lower figures were found for other age groups. These percentages should not be taken as a definitive measure of the degree of under-reporting as the technique is only undertaken in a small, and possibly unrepresentative, sample of NDNS participants due to the high cost of the isotope. Due to the intensive nature of the survey, it is not carried out at the same time as the diary recording energy intake. Although direct comparison of energy intake with energy expenditure is not possible, the study nevertheless gives a clear indication that there is substantial under-reporting in NDNS and also appears to suggest a slightly higher degree of under-reporting than was indicated by results of other DLW studies carried out for previous NDNS surveys in the 1990s. However methodological differences make it difficult to be sure that comparisons are reliable.

It is plausible that increased under-reporting is a factor in the decline in reported energy intakes in NDNS. Changes in eating behaviour recorded in dietary surveys such as an increase in snacking behaviour, more food eaten outside the home and the decline of the ‘three meals a day’ eating pattern could have led to more under-reporting as foods eaten outside meal times or outside the home are more prone to being omitted from records. Identifying the composition and estimating portion size can be more challenging for foods eaten out. Food purchase surveys show that changing lifestyles have led to less reliance on a single weekly shop and a higher proportion of purchases made through top-up shopping trips by members of the household other than the main shopper. All these factors could well have contributed in an increase in under-reporting over time. However there is very limited evidence available to confirm this or to allow an assessment of the scale of the increase.

Two doubly labelled water studies were carried out as part of NDNS in 2000 and 2008/12. The latter study reports a higher average proportion of calories underestimated than the former (32% on average compared with 25%) but a number of methodological differences between the studies and the small, unrepresentative subsamples included in the studies mean that we cannot be confident that a small observed difference represents a real increase in under-reporting. It should also be noted that there was a wide variation between individuals in the degree of under-reporting. Another study from the 1990s showed 19% of calories underestimated but this was from a research study unrelated to NDNS so cannot be compared directly with the NDNS DLW studies.

The DLW technique is the most accurate method available for assessing energy expenditure but does not tell us anything about what people are actually eating or their nutrient intakes. It simply allows an assessment of the extent to which the energy intake an individual reports in a dietary survey like NDNS is likely to reflect their actual or usual energy intake – for healthy adults energy intake should balance energy expenditure over a period of time if they are maintaining a constant weight. The DLW method cannot distinguish the source of energy expended (e.g. if it is from fat or carbohydrate, from meat or cereals) and so cannot be used to calibrate or correct food consumption reported in the diary or the nutrient intakes derived from those.

Nôl i'r tabl cynnwys

.Annex C: Overview of data collection methods and the derivation of nutrient intake from the Living Costs & Food Survey

The figures in Family Food are sourced from The Living Costs and Food Survey run by the Office for National Statistics. One element of the survey, The Family Food Module, collects detailed quantity and expenditure information on household food and drink purchases as well as itemised lists of food and drink consumed outside the home for use by DEFRA.

The Office for National Statistics has overall project management and financial responsibility for the survey while DEFRA sponsors the specialist food data.

Each individual aged 16 and over in the household is asked to keep diary records of daily expenditure for two weeks. Although the diaries are expenditure based, quantities are recorded where possible for household purchases of food and drink. Where expenditure is recorded for meals eaten out, the diary keeper records an itemised list of the meal components. Where quantities are not recorded on the diary they are estimated using standard portions sizes.

The ‘household’ category covers all food that is brought into the household. ‘Eating out’ covers all food that never enters the household such as: restaurant meals, school meals and snacks bought and eaten away from home. Diary entries relating to food and drink are coded into roughly 500 different categories of food and drink covering household purchases (about 250) and eating out purchases (about 250).

Estimation of Nutrient Intakes

Estimated nutrient intakes are calculated from food purchases using nutrient composition data supplied by Public Health England (PHE). The majority of the data is from PHE’s nutrient analysis programme, supplemented by values from manufacturers and retailers. Each of the 500 food codes in the Family Food Module is made up of a number of sub-codes with nutrient composition data attached. For example, the food code ‘fruit juice’ comprises a number of sub-codes including grapefruit, orange, pineapple and apple juices. A weighted average nutrient composition is calculated for each food code based on estimates of the market share of each sub-code. Nutrient values for the food codes in the eating out component of the survey are generated by mapping each code to an equivalent food code or combination of food codes in the nutrient databank for the National Diet and Nutrition Survey.

The nutrient composition data is updated on a rolling basis to keep information in line with new or reformulated products. All nutrient compositions are based on edible food and take into account inedible (unavoidable) waste, e.g. banana skins.

Nôl i'r tabl cynnwys

Manylion cyswllt ar gyfer y Methodoleg

Jen Woolford
obesity@ons.gov.uk