1. Methodology background


 National Statistic   
 Survey name  Wealth and Assets Survey
 Frequency  Biennial
 How compiled  Longitudinal survey
 Geographic coverage  Great Britain, excluding addresses north of the
Caledonian Canal, the Scottish Islands and the
Isles of Scilly
 Sample size  Approximately 30,000 households in wave 1, 20,000 in wave 2, and 21,000 in wave 3, 20,000 in Wave 4, 18,000 in Wave 5.
 Last revised  29 January 2018

Nôl i'r tabl cynnwys

2. Executive summary

The Wealth and Assets Survey (WAS) launched in 2006 and is a biennial longitudinal survey conducted by Office for National Statistics (ONS). This survey measures the well-being of households and individuals in terms of their assets, savings and debt and planning for retirement. The survey also examines attitudes and attributes related to these. Data from this longitudinal survey will also provide users with the ability to measure changes of wealth in Great Britain over time. The survey is currently sponsored by a funding consortium, including ONS, Department for Work and Pensions (DWP), HM Revenue and Customs (HMRC), Financial Services Authority (FSA) and the Scottish Government (SG). Interviewers working on the survey refer to it as the Household Assets Survey (HAS).

Approximately 30,000 households were interviewed in wave one, 20,000 in wave two, 21,000 in wave three, 20,000 in wave 4 and 18,000 in wave 5. As wealth is known to be unevenly distributed, addresses more likely to contain wealthier households were sampled at a higher rate to improve the efficiency of the sample. These addresses were identified utilising data from HMRC.

The results of the Wealth and Assets Survey are used by DWP, HMRC, ONS and other government departments (including the Department for Business, Energy and Industrial Strategy (BEIS) and Department of Health (DH)), as well as academics. The data provides a greater understanding of the levels and distribution of wealth in terms of pensions, property, financial and physical assets as well as indebtedness.

The datasets for waves one to five have been released to consortium members as well as the UK Data Archive (UKDA) under Special Licence and to the Virtual Microdata Laboratory (VML) managed by ONS. Due to the need to maintain respondents’ anonymity, certain variables have not been released to consortium members and the UKDA. To encourage the widest possible use of the data, including internationally, an End User Licence (EUL) dataset was created and deposited with the UKDA in January 2018.

This survey remains valuable as the WAS survey provides data on topics that are not sufficiently covered elsewhere.

This report contains the following sections:

  • Output quality

  • About the output

  • How the output is created

  • Validation and quality assurance

  • Concepts and definitions

  • Other information, relating to quality trade-offs and user needs

  • Sources for further information or advice

Nôl i'r tabl cynnwys

3. Output quality

This report provides a range of information that describes the quality of the output and details any points that should be noted when using the output.

We have developed Guidelines for Measuring Statistical Quality; these are based upon the five European Statistical System (ESS) quality dimensions. This report addresses these quality dimensions and other important quality characteristics, which are:

  • relevance
  • timeliness and punctuality
  • comparability
  • coherence
  • accuracy
  • output quality trade-offs
  • assessment of user needs and perceptions
  • accessibility and clarity

More information is provided about these quality dimensions in the following sections.

Nôl i'r tabl cynnwys

4. About the output

Relevance

(The degree to which the statistical outputs meet users’ needs.)

A number of government departments joined the Wealth and Assets Survey (WAS) consortium because this survey was identified as being able to supply data on topics that were not sufficiently covered elsewhere. This survey fills a major information gap on wealth and indebtedness at a household and personal level. The pension wealth data in particular is unique due to its detail. The longitudinal element provides a further dimension to this dataset allowing users to analyse levels of change across all waves, from lower levels of wealth and indebtedness to households or individuals’ total wealth.

The survey has a large sample and almost complete coverage of Great Britain. The results of wave one to wave 5 have been and are likely to be used by the Department for Work and Pensions (DWP), HM Revenue and Customs (HMRC) and other government departments, as well as analysts within Office for National Statistics (ONS) and academics to provide a greater understanding of the levels and distribution of wealth in terms of pensions, property, financial and physical assets, as well as indebtedness.

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.)

As the survey has been in existence for a relatively short period and has a biennial interview wave pattern, data for the first five waves have been released so far.

These data are available in the following releases:

For more details on related releases, the Release calendar is available online and provides 12 months’ advance notice of release dates. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Official Statistics.

Building on the investment of work through during wave one and in particular the wave 2 process, the ONS and the consortium are improving the efficiency of the data derivation, cleaning and validation process so that the publication of datasets for wave six and beyond should be more timely.

Nôl i'r tabl cynnwys

5. How the output is created

This longitudinal Wealth and Assets Survey (WAS) measures the numbers and values of assets, debt and savings as well as attitudes to savings and indebtedness. Classificatory variables (age, sex, employment status) are also covered. The first wave of the survey commenced with interviews carried out over two years from July 2006 to June 2008. A second wave took place two years on from initial interviews, covering the period July 2008 to June 2010, and a third wave began in July 2010 and was completed in June 2012. A fourth wave of WAS commenced in July 2012 and completed in June 2014. A fifth wave of WAS commenced in July 2014 and completed in June 2016.

The WAS is a continuous survey with interviews spread evenly over the year, which helps to ensure that estimates are not biased by seasonal variations.

The survey samples private households in Great Britain, excluding north of the Caledonian Canal, the Scottish Islands and the Isles of Scilly.

The stratification of the sample for the first wave of WAS was based on regional and census 2001 variables and had two stages. At the first stage, a stratified sample of primary sampling units (PSUs) was drawn from the small users postcode address file (PAF). Postcode sectors were used as PSUs. The second-stage involved 26 addresses per PSU being sampled using systematic random sampling. The list of addresses was sorted by postcode and street number, the proportion of households with the household reference person (HRP) in National Statistics Socio-economic classification (NS-SEC) group one to three, and also the proportion of households without a car. This stratification is using judgement based on experience from optimising stratifiers for the Family Resources Survey (FRS) and Living Costs and Food Survey (LCF).

The NS-SEC stratifier is the most powerful stratifier for economic social surveys. Car ownership was chosen over economic activity for the second stratifier because this is more correlated with wealth. The sampling was carried out in such a way that the addresses flagged as expected to feature wealthier households had two and a half to three times the probability of being sampled as non-flagged addresses.

For the first two years of the first wave of the survey, 1,200 PSUs were drawn, giving a set sample of 31,200 addresses per year.

For the second and subsequent waves, all households that responded in the first wave were revisited, as well as all households that could not be contacted in the previous wave. To ensure respondents’ contact details are maintained between waves, a “keep in touch” phone call is administered approximately four months prior to the respondent’s next interview. This exercise can document households that have split and those that have moved.

As the sample for each subsequent wave consists predominantly of the preceding wave’s respondents, the size of the sample reduces with each wave. To mitigate the effect of attrition, a new cohort was introduced in wave 3 (8,000 new addresses in year one and 4,000 in year two). A further cohort of 8,000 addresses was introduced in wave four, and a further cohort of 6,000 in wave 5. The new cohort improves the size of the cross-sectional sample, which is required because attrition has reduced the sample since wave 1. The new cohort then may help to reduce any bias introduced by attrition as the new cohort is selected from the current population and hence helps in accounting for changes in the characteristics of population over time.

For further details see Chapter 10, Technical details, in the Wealth in Great Britain, Main Results from the Wealth and Assets Survey 2006 to 2008 and Chapter 4 in the Wealth in Great Britain Wave 2 2008 to 2010 (Part 1).

Editing

An extensive range of validation checks and computer edits were applied to both the household and individual questionnaires during the Computer Assisted Personal Interview (CAPI) and to the aggregate data file in the office.

Imputation

Imputation is an adjustment process that is used to determine and assign replacement values to resolve problems of missing, invalid or inconsistent data.

The problem of missing data in WAS is approached in two stages: firstly, a deductive imputation method, followed by a statistical method. Deductive imputation was applied where a missing or inconsistent value could be deduced with certainty. Secondly, statistical imputation was carried out using a nearest-neighbour imputation method where information from a donor record that had no errors or missing values was used to replace the missing values for a recipient record. In this approach a donor is selected from a pool of potential donors with similar characteristics based on conditional probabilities. For longitudinal households, where an observed value is present in one wave but the other wave is missing and therefore requires imputation, an imputed value is drawn from a donor with reference to the observed value or calculated based on observed relationships or ratios between variables in the donor record. The imputation is conducted under edit constraints to ensure that outliers and implausible relationships are not introduced into the data through the imputation process.

Outliers

As part of the data cleaning process, cross-sectional outliers were identified on all monetary variables used to compile derived variables. Large changes between waves were also identified as longitudinal outliers. Outlier thresholds were determined through analysis of the distribution of the data. Each variable was analysed and dependent upon the nature of that variable and the spread of its data, a percentage of the highest and lowest values were identified as outliers. Not all variables had their lowest values labelled as outliers as low values can be acceptable for some variables, for example, zero values in financial accounts.

Outliers were checked for credibility through examination of other variables, including the previous waves’ responses, in an attempt to find evidence to support or inform an edit to the outlier. This evidence includes the inspection of wealth, through income, assets and debts and verification from linked variables (comparisons of mortgage value with monthly mortgage payment and remaining term). There are reasons to justify substantial longitudinal changes. Alterations to working status or household structure, in particular a split in partnership or a house move, can significantly affect the longitudinal change of many variables.

Amendments were only made to data where sufficient evidence to support an amendment existed. In waves one and two approximately 5% of the data were investigated as outliers, of which a minority of these were amended.

A more systematic approach was established for the identification of cross-sectional and longitudinal outliers from wave three onwards.

Weighting

A three-stage weighting procedure was implemented in WAS. First, a design weight, equal to the reciprocal of the address selection probability, was constructed. Secondly, a non-response weight was created to compensate for non-response bias. This was created using information on respondents and non-respondents taken from sources including the Financial ACORN classification system. This applies to a new cohort. In older panels an attrition adjustment is applied, joiners are incorporated, before calibration.

The final stage of the weighting procedure calibrated the product of the design and non-response weights to known population totals taken from official population estimates present at the time of the fieldwork period. An integrative calibration technique was used to ensure that individuals in the household each received the same weight, which was also the household weight.

As analysis can be performed longitudinally and cross-sectionally on the data, two individual weights were created to facilitate the analysis required.

More information on the methods used to compile the output can be found in Chapter 10, Technical details, in the Wealth in Great Britain, Main Results from the Wealth and Assets Survey 2006 to 2008 and Chapter 4 in the Wealth in Great Britain Wave 2 2008 to 2010 (Part 1).

Statistical disclosure control methodology is applied to WAS data. This ensures that information attributable to an individual or individual organisation is not identifiable in any published outputs. The Code of Practice for Official Statistics and specifically the Principle on Confidentiality set out practices for how we protect data from being disclosed. The Principle includes the statement that ONS outputs should “ensure that official statistics do not reveal the identity of an individual or organisation, or any private information relating to them, taking into account other relevant sources of information”.

More information can be found in the National Statistician’s Guidance: Confidentiality of Official Statistics and also on the Disclosure Control Policy for Social Survey Microdata page.

Nôl i'r tabl cynnwys

6. Validation and quality assurance

Accuracy

(The degree of closeness between an estimate and the true value.)

Multiple quality assurance methods ensure that the Wealth and Assets Survey (WAS) data are as reliable as possible. These methods are applied during the interview and post collection through outlier detection and comparisons of the data between waves. All data that are identified as possible errors are investigated and where appropriate altered.

Revisiting respondents in subsequent waves provides the opportunity to confirm some of the previous waves’ data. This is of particular importance for respondents whose previous waves’ interview was given by proxy. Comparisons of data between waves may lead to revisions to improve the accuracy of previously published data.

Sampling error

All reasonable attempts have been made to ensure that the data are as accurate as possible. However, there are two potential sources of error that may affect the accuracy of estimates and for which no adequate adjustments can be made: these are known as sampling and non-sampling errors.

Sampling error refers to the difference between the results obtained from the sample and the results that would be obtained if the entire population was fully enumerated. The survey estimates are therefore likely to differ from the figures that would have been produced if information had been collected for all households or individuals in Great Britain. The extent to which survey estimates vary from their population values can be estimated, to a given level of confidence, through the calculation of confidence intervals via the standard error of the estimate.

The standard error is a measure of sampling variability, which shows the extent to which the estimates are expected to vary over repeated random sampling. To estimate standard errors correctly, the complexity of the survey design needs to be accounted for.

Some estimates of standard errors for main variables are available in the supporting tables in Chapter 10, Technical Details Main Results from the Wealth and Assets Survey 2006 to 2008. However, these standard error estimates do not account for imputation, which may affect variability.

Additional inaccuracies, which are not related to sampling variability, may occur for reasons such as errors in response and reporting. Inaccuracies of this kind are collectively referred to as non- sampling errors and may occur in a sample survey or a census. The main sources of non-sampling error are:

  • response errors such as misleading questions, interviewer bias or respondent misreporting

  • bias due to non-response as the characteristics of non-responding persons may differ from responding persons

  • data input errors or systematic mistakes in processing the data

Non-sampling errors are difficult to quantify in any collection. However, every effort was made to minimise their effect through careful design and testing of the questionnaire, training of interviewers and extensive editing and quality control procedures at all stages of data processing. Imputation is another method used to improve accuracy resulting from missing observations in the dataset.

Response

Response rates are reported on a monthly basis and are based on the number of fully and partially co-operating households as a proportion of the numbers of eligible households in the sample. A response rate of 55% was achieved for wave 1 and 68% of the eligible households’ sampled responded in wave 2. In wave three the response achieved for the new cohort was 51% and 73% for the old cohort. In wave 4 the response achieved for the new cohort was 53% and 70% for the old cohort. In wave 5 the response achieved for the new cohort was 55% and 69% for the old cohort.

Regional response rates for WAS have not varied a great deal, although London, in common with other social surveys, does tend to exhibit markedly lower response rates.

Comparability and coherence

(Comparability is the degree to which data can be compared over time and domain, for example, geographic level. Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar.)

Major government surveys now use harmonised questions on important topics to ensure comparability of results. Where appropriate, WAS questions are harmonised with other government surveys. A list of harmonised questions is available via the Harmonisation Programme.

Separate datasets for each wave are issued after all checks have been completed. Although many of the variables are comparable between waves, some datasets have changed as have some of the categories of responses for particular variables. When this occurs, details are provided in the user guides and variable lists.

A glossary of the main terms used in the Wealth and Assets Survey is provided in the Glossary: Wealth in Great Britain 2006 to 2008.

There is limited comparable data from administrative sources or major surveys for some topics covered by the survey. Nevertheless, using information that was available; the comparability of the results with results from other sources have been checked by the various contributors to the report as part of the validation process. This comparison will have included information from less extensive surveys, administrative data and the Financial Reporting Standards (FRS).

Nôl i'r tabl cynnwys

7. Concepts and definitions

(Concepts and definitions describe the legislation governing the output, and a description of the classifications used in the output.)

The classifications used for the Wealth and Assets Survey (WAS) are harmonised with other government surveys. These classifications are:

  • Household Outcome Code

  • UK Standard Industrial Classification (UK SIC)

  • UK Standard Occupational Classification (UK SOC)

  • Country of birth

  • Nationality

  • Religion

  • Ethnicity

Nôl i'r tabl cynnwys

8. Other information

Output quality trade-offs

(Trade-offs are the extent to which different dimensions of quality are balanced against each other.)

There is a trade-off between accuracy and timeliness of data dissemination. In theory, the Wealth and Assets (WAS) data could be disseminated immediately after fieldwork completion. However, the format of the data and level of item or unit non-response would significantly reduce the analytical value and usability of the data.

We have taken the decision to undertake editing and imputation of WAS data prior to its dissemination. This significantly improves the quality and usability of data available for analysis; but does delay the dissemination of data. We are actively working to reduce the amount of time that the edit and imputation stages take, to retain accuracy whilst ensuring the data are disseminated as soon after data collection as possible.

The longitudinal nature of the survey allows us to validate the responses provided in the previous interview at the current interview. For example, on occasion information is provided by proxy at one wave and then in person at the subsequent wave. Personal responses are considered to be more accurate and therefore we take the opportunity to improve the quality of the previous wave’s responses. This can mean that higher quality data are available at a later date; which can lead to the revision of previously published estimates.

Assessment of user needs and perceptions

(The processes for finding out about users and uses, and their views on the statistical products.)

The survey questionnaire is developed in full consultation with major stakeholders and received overwhelming support from a public consultation that was carried out in 2010 and 2011. The study of users and their use of WAS data is essential for improving our understanding and is therefore constantly examined through a number of different procedures.

We engage with users through the WAS User Group, which meets to discuss their needs and views. This provides us with the opportunity to receive and react to feedback and also presents users with the opportunity to network and share their analysis. The use of WAS data is continuously monitored through the approved researcher special licence applications received from the UK Data Archive (UKDA).

During June 2012, ONS conducted a customer satisfaction survey. Registered users that participated included University of St Andrews, HM Treasury (HMT), Department for Work and Pensions (DWP), Institute for Fiscal Studies (IFS), Institute for Public Policy Research (IPPR), Lancaster University and Scottish Government. Participants were asked to comment on potential improvements to the questionnaire content and supporting documentation as well as their purpose for the data.

The consultation illustrated the extensive analysis performed by a wide range of users and also the uniqueness of WAS data across a broad scope of wealth types. Supporting documentation was highlighted as an area requiring improvement, resulting in the expansion of the User Guide to a portfolio containing six volumes, available in the Wealth and Assets Survey – User Guidance.

Nôl i'r tabl cynnwys

9. Sources for further information or advice

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

The UK Data Archive (UKDA) at Essex University provides access to approved researchers under special user licence.

Documentation to guide the users of these datasets has been provided to the consortium members and is available to UKDA approved researchers.

Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. We also offer users the option to download the narrative in PDF format. In some instances other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information, please contact us via email at wealth.and.assets.survey@ons.gsi.gov.uk.

More information regarding conditions of access to data is available:

Nôl i'r tabl cynnwys