Development of regional household expenditure measures

• Today (26 September 2018) we have published experimental regional estimates of household spending across the whole UK for the first time, aimed at showing users what is possible; the production of these estimates has involved making some very broad assumptions using currently available data sources, some of which have limited sample sizes, and so strong caution is advised when interpreting the findings.

• Over the next few years, we aim to identify and introduce new data sources that will allow us to improve the quality of these experimental figures and further understand how changes in sampling and the assumptions made can affect the results; we will use these initial results to consult with users on how best we can develop them in the future.

• London had the highest national expenditure per person in 2016, at £24,545, mainly driven by the higher housing costs in and around the capital; the lowest spending per person in 2016 was seen in the West Midlands, at £15,276.

• In terms of growth in spending per person between 2015 and 2016, the North East had the greatest increase, at 8.1%; this growth was seen across a wide range of goods and services, with the strongest growth seen in the household goods and services, and clothing and footwear categories.

• The lowest growth in spending per person between 2015 and 2016 was seen in Northern Ireland, at negative 0.4%, the only country or region to see a fall in spending per person in this period.

• The households’ saving ratio is the percentage of total resources left after all spending has occurred; it varies considerably across the countries and regions of the UK, with saving in London and the West Midlands being the highest in 2016, at 14.5% and 12.8% respectively.

• The lowest levels of saving in 2016 were seen in the South West, at 1.5%, followed by Northern Ireland and Wales, at 2.5% and 2.6% respectively; these figures compare with a UK average saving ratio of 6.9%.

With increased devolution of powers to local and combined authorities within the UK, the need for statistics to monitor and inform policy at a regional level has also been increasing to an unprecedented level. Users of regional statistics have been telling us for years that they need more and better data, and the recent review of economic statistics by Sir Charles Bean recommended that more should be done to provide statistics for smaller areas within the UK.

We have responded to this need by setting up a Devolution Project, with around a dozen separate work streams designed to develop and provide the statistics needed by regional and local users. The project is funded until 2020 and includes plans to deliver many new and improved statistics, details of which can be found in the Economic Statistics and Analysis Strategy and in an article describing the aims of the project.

One of these work streams is to develop a regional measure of household final consumption expenditure, hereafter referred to as household expenditure or by the abbreviation HFCE. There are many user needs that can be met through this development, including:

• information on the spending habits of householders, allowing better planning of facilities and infrastructure by local government

• improved investment planning by businesses providing goods and services

• expansion of the household account at a regional level, allowing the derivation of the saving ratio for subnational areas (a useful indicator of prosperity)

• completion of the European Union’s European System of Accounts 2010: ESA 2010 regional transmission tables (including all voluntary requirements), making the UK fully compliant with the regulation

At a regional level, we currently measure the income and outgoings of households only as far as gross disposable household income (GDHI). That measures the primary income components, such as wages and salaries and property income, and the secondary distribution of income, such as the effect of taxes on income and social benefits. GDHI is a measure of the amount of money people in households have available for spending or saving.

Regional household expenditure takes this to the next stage by measuring how much money people in households spend on each of a range of commodities. Once all spending is accounted for, we are left with a measure of saving.

Although these experimental estimates are the first regional measures produced on a consistent basis across the whole UK, the devolved administrations of Scotland and Northern Ireland have been independently compiling and publishing estimates of household expenditure for their respective countries for several years. While our estimates are still in an early stage of development, you are advised to give more credence to the official Scottish and Irish estimates.

In the UK National Accounts, household final consumption expenditure (HFCE) is measured using an approach known as the domestic concept, whereby all money spent in the UK on a particular commodity is measured, regardless of who is doing the spending. It therefore includes spending by foreign visitors, but it excludes spending abroad by UK residents on holiday or business.

The total spent on all commodities in the UK is then adjusted to remove the spending by foreign visitors and add the spending by UK residents abroad to give a total for all spending by UK residents, which is known as the national concept. This adjustment is only done at a total level, not for each commodity separately.

When we come to regional measures, the approach used in the national accounts runs into a problem. Using the domestic concept, we can measure the amount spent in each region. But if we only adjust that figure to account for international spending, we will not take into account spending by residents of one region in another region of the UK. When you consider that every time you spend money in a place other than your home town you are effectively spending “abroad” at a regional level, you can imagine how big a problem this can be.

To overcome this we have to measure regional spending using both the domestic and the national concepts. We therefore measure all the spending that takes place in a region, regardless of where the person spending comes from, and we measure separately all the spending by the residents of a region, regardless of where they are when the spending takes place. We still need to account for international spending, and for this we also need to break down that spending into commodities.

An unexpected by-product of this approach is that having the two independent measures allows us to estimate the net inter-regional spending (or trade) flow for each region from the difference between the two measures (at least for the households sector). This is quite a bonus, since until now it has always been considered impractical to attempt to measure inter-regional trade.

In the tables published with this article, you can find our estimates of this net spending between regions, calculated as domestic expenditure minus foreign spending in the UK, minus national expenditure, plus spending abroad by UK residents.

Geography

The geographic areas for which we provide regional statistics are based on the Nomenclature of Units for Territorial Statistics (NUTS), a classification of geographic areas used across the European Union to provide a consistent framework for regional accounts. The NUTS areas are updated every three years to reflect changes to administrative boundaries and react to variations in population growth. The latest set of NUTS areas came into effect on 1 January 2018.

For the UK, the NUTS areas are currently:

• NUTS1: Scotland, Wales, Northern Ireland and nine English regions
• NUTS2: 41 sub-regions – mostly groups of counties and unitary authorities
• NUTS3: 179 local areas – mostly single counties and unitary authorities

In addition, we are trying to be more responsive to emerging and changing user needs for statistics relating to different geographic areas. In recent years, the establishment of local enterprise partnerships (LEPs) across England and combined authorities covering city regions has increased the demand for more flexible geographic statistics.

Where possible we now try to provide figures for the 400 local authority districts of the UK in addition to the NUTS areas. Most of the new areas can be constructed by aggregating local authorities together, so this provides a framework that can be used to widen the scope of regional estimates and meet a lot more user needs.

Commodities

The classification we use for the various goods and services that people spend their money on is called the Classification of Individual Consumption by Purpose (COICOP). The classification has three levels: divisions (two digit); groups (three digit); and classes (four digit).

The UK National Accounts presents figures for all three levels across most of the commodities, but for regional accounts the limitations of the data sources we have available mean that we would need to stretch the data rather thinly to achieve such a detailed breakdown, and the quality of results could be adversely affected by this. Therefore, we have chosen to provide mostly a group-level breakdown, with the exceptions being for education, where only a division-level measure is provided even for the UK as a whole, and for some groups where the classes are sufficiently distinct and the data available are deemed good enough to support the extra detail.

Table 1 shows the commodity breakdown we have chosen to produce at a regional level.

In the consultation that follows the publication of this article, we would like to hear from users if this level of commodity detail meets their needs and, if not, where additional detail would be useful. We would also like to hear about which parts are of most interest to users of smaller area statistics, as it is unlikely that we will be able to provide as much detail for smaller geographies.

The principal data sources we have identified that are currently available for use in the regional allocation of household expenditure are two Office for National Statistics (ONS) surveys: the Living Costs and Food Survey (LCF) and the Annual Business Survey (ABS).

The LCF is a survey of households that collects information about people’s spending habits on a residential basis, that is, the data are allocated to the regions where people live. As such, it provides data appropriate for the national concept measure of household expenditure.

The LCF collects information from households via an interview and through the completion of a diary in which householders record their spending over a two-week period. The data are therefore extremely detailed, which is a strength. However, the LCF does also have a weakness; its total sample size, of around 5,000 households per year, is rather small to provide a good representation across all regions of the UK, particularly at smaller geographic levels. It also excludes people living in communal establishments, whose spending patterns may not be the same as those living in conventional households.

The ABS is a survey that collects information from businesses, including retail sales of various commodities at the point of sale, that is, the data are allocated to the regions where the spending takes place. It therefore provides data appropriate for the domestic concept measure of household expenditure. For commodities where retail sales information is not directly collected, mainly for the provision of services, we have attempted to identify businesses, classified according to the Standard Industrial Classification 2007: SIC 2007, where the nature of the activity provides a good match to the commodity of interest. For these commodities we have used the total turnover of the businesses in a region as the variable that guides the regional allocation of expenditure.

The ABS has strength in its coverage of businesses across the UK, as it has a large sample size of around 80,000 businesses per year and collects detailed information. Its main weakness for our use lies in the fact that there is no way to distinguish between sales to households and sales to business. We therefore need to make the assumption that the proportion of total sales that represents business use is equal across all regions of the UK. This may or may not be a valid assumption.

Between them, these two surveys provide data for most of the commodities we want to cover, across both domestic and national concept measures. There are gaps, though, for which we need to use alternative data sources or, if none can be found, modelling approaches will be needed to provide a complete picture.

In this section we describe each Classification of Individual Consumption by Purpose (COICOP) division in terms of how well we have been able to obtain appropriate regional data, and where we have encountered issues requiring alternative solutions.

COICOP 01: Food and soft drinks

For this division we have good coverage of all components on both conceptual bases. For the domestic concept, we have ABS retail sales by commodity with a class-level breakdown for food and group-level data for beverages. For the national concept, we have LCF data with a class-level breakdown for both food and drink.

COICOP 02: Alcohol, tobacco and narcotics

For this division we have good coverage of alcoholic beverages and tobacco but poor coverage of narcotics. For the domestic concept, we have ABS retail sales by commodity with a group-level breakdown for alcoholic drinks and tobacco. For the national concept, we have LCF data with a class-level breakdown for the same groups. However, it is recognised that there is a tendency for households to under-report their use of alcohol and tobacco, which could affect the quality of the data collected.

We have no regional data on spending on narcotics. We therefore have used the same method we use to allocate illegal drugs in our regional gross value added measures, which is to use the regional distribution of adult population numbers. This means the regional allocation of narcotics merely reflects the overall population and contains no true information about regional variation in drug use.

COICOP 03: Clothing and footwear

For this division we have good coverage for the national concept, with LCF data for all components at the class-level. For the domestic concept, we have ABS retail sales by commodity for some class-level components, but not for those relating to cleaning, hire and repair of clothing and footwear or for clothing materials. For these we have used the regional distribution of ABS turnover for businesses classified to SIC 2007 codes: 47.51 (Retail sale of textiles in specialised stores); 95.23 (Repair of footwear and leather goods); and 96.01 (Washing and dry-cleaning of textile and fur products).

COICOP 04: Housing

For this division we have a rather more varied coverage of commodities by both concepts. For actual and imputed rental we have good quality data compiled at the NUTS1 level that are used in the UK National Accounts measure. These use source data from the Valuation Office Agency (VOA) and the devolved administrations of Wales, Scotland and Northern Ireland. We do not have good quality data on the ownership of second homes across the regions of the UK. Because the levels of spending relating to second homes are very small, we have combined them with primary homes for the regional allocation and used the same source data for both.

For the national concept, we have LCF data at class-level for the other components. However, we also have administrative data on the consumption of various fuels by households from the Department for Business, Energy and Industrial Strategy (BEIS). These administrative data do not suffer from the LCF’s small sample size and are mostly available with a local authority breakdown. This makes it our preferred choice for household fuels: electricity; gas; liquid fuels; and solid fuels.

For the domestic concept, we have ABS retail sales by commodity for decorating and DIY supplies, but for most other components we have used ABS turnover for businesses classified to SIC 2007 codes:

• 35.1 (Electric power generation, transmission and distribution)
• 35.2 (Manufacture of gas; distribution of gaseous fuels through mains)
• 36 (Water collection, treatment and supply)
• 37 (Sewerage)
• 38.1 (Waste collection)
• 43.2 (Electrical, plumbing and other construction installation activities)
• 43.3 (Building completion and finishing)

We have no data for the domestic concept on liquid and solid fuels, but for these commodities we believe it is reasonable to make the assumption that most spending takes place close to the home, from local suppliers. We therefore propose to use the BEIS data on consumption of liquid and solid fuels for both national and domestic concepts.

There are no UK-level data on expenditure on heat energy (COICOP 04.5.5) so we have excluded this code from the regional measures.

COICOP 05: Household goods and services

For this division we have good coverage of most components on both conceptual bases. For the national concept, we have LCF data with a class-level breakdown across all components. For the domestic concept, we have ABS retail sales by commodity with mostly a group-level breakdown, plus some class-level components, such as furniture, carpets and non-durable household goods.

For the domestic concept, we have used ABS turnover for the components relating to the repair of furniture, floor coverings and household appliances. For these we use businesses classified to SIC 2007 codes: 95.22 (Repair of household appliances and home and garden equipment); and 95.24 (Repair of furniture and home furnishings). We also have no domestic concept data on domestic and household services, so here we make the reasonable assumption that all such activity takes place in the home and we can use the corresponding LCF data for both concepts.

For the national concept, the LCF data relating to the repair of furniture and floor coverings (COICOP 05.1.3) are too sparse to provide reliable estimates, even at the country and region level. For this commodity we make the assumption that most such repairs use local suppliers, and we have therefore used the ABS turnover data for SIC code 95.24 (Repair of furniture and home furnishings) for both domestic and national concepts.

COICOP 06: Health

For this division we have fairly good coverage for the national concept, with LCF data for all components at the class-level, although the sample sizes achieved are rather small even by LCF standards. For the domestic concept, we have ABS retail sales by commodity for some components, but not for those relating to out-patient and hospital services. For these we have used the regional distribution of ABS turnover for businesses classified to SIC 2007 codes: 86.1 (Hospital activities); and 86.9 (Other human health activities). It should be noted that these will generally only reflect spending on private healthcare and not that relating to NHS healthcare.

For the national concept, the LCF data relating to hospital services are too sparse to provide reliable estimates, even at the country and region level. Here we make the assumption that most people will use local hospitals and we have used the corresponding ABS turnover data for both concepts.

COICOP 07: Transport

For this division we have good coverage for the national concept, with LCF data for all components at the class-level. However, some of the LCF data reflect spending on combined fares by different modes of transport, which makes it more difficult to separate the specific types of transport services. For this reason, we have chosen not to attempt to split the group-level data for transport services.

For the domestic concept, we have ABS retail sales by commodity for spare parts, vehicle fuel and bicycle sales, but not for any other components. For these we have used the regional distribution of ABS turnover for businesses classified to SIC 2007 codes:

• 45.11 (Sale of cars and light motor vehicles)
• 45.2 (Maintenance and repair of motor vehicles)
• 45.4 (Sale, maintenance and repair of motorcycles and related parts and accessories)
• 49.1 (Passenger rail transport, interurban)
• 49.31 (Urban and suburban passenger land transport)
• 49.32 (Taxi operation)
• 49.39 (Other passenger land transport)
• 49.42 (Removal services)
• 50.1 (Sea and coastal passenger water transport
• 50.3 (Inland passenger water transport)
• 51.1 (Passenger air transport)
• 77.11 (Renting and leasing of cars and light motor vehicles)
• 79.1 (Travel agency and tour operator activities)
• 85.53 (Driving school activities)

The COICOP class 07.2.4 (Other vehicle services) presents us with some particular measurement issues, as it covers a wide and varied range of activities including: vehicle rental and leasing; garage hire; road and bridge tolls; parking charges; driving lessons and licences; and MOT test fees. For the national concept, we have LCF data for this category as a whole, but for the domestic concept we have attempted to piece together sufficient components to provide a decent representative measure. Of the items listed, we have ABS turnover for vehicle rental and leasing, and for driving lessons. We have also obtained data on parking revenue collected by local authorities, from the Ministry of Housing, Communities and Local Government (MHCLG) and the devolved administrations of Wales, Scotland and Northern Ireland. Between them, we believe these data provide a sufficient level of coverage of the major items in this category.

COICOP 08: Communication

For this division we have LCF data with a group-level breakdown for the national concept. For the domestic concept, we have ABS retail sales by commodity for telephone and telefax equipment. For the other components we have used ABS turnover for businesses classified to SIC 2007 codes: 53.1 (Postal activities under universal service obligation); and 61 (Telecommunications). We chose not to include SIC 2007 code 53.2 (Other postal and courier activities) as we believe this to be dominated by business use rather than by households.

COICOP 09: Recreation and culture

For this division we have good coverage for the national concept, with LCF data at the class-level for all components. For the domestic concept, we have ABS retail sales by commodity for many components, at either the group or class level. For other components we have used the regional distribution of ABS turnover for businesses classified to SIC 2007 codes:

• 30.12 (Building of pleasure and sporting boats)
• 33.15 (Repair and maintenance of ships and boats)
• 33.16 (Repair and maintenance of aircraft and spacecraft)
• 45.19 (Sale of other motor vehicles)
• 45.2 (Maintenance and repair of motor vehicles)
• 74.2 (Photographic activities)
• 75 (Veterinary activities)
• 85.51 (Sports and recreation education)
• 85.52 (Cultural education)
• 90.01 (Performing arts)
• 90.04 (Operation of arts facilities)
• 91 (Libraries, archives, museums and other cultural activities)
• 92 (Gambling and betting activities)
• 93 (Sports activities and amusement and recreation activities)
• 95.1 (Repair of computers and communication equipment)
• 95.21 (Repair of consumer electronics)
• 96.04 (Physical well-being activities)

Because we have needed to combine some of these measures to provide coverage of the range of commodities within certain COICOP classes, we have in some cases reduced the weight given to particular measures that include activity that is out of scope for our use. The repair and maintenance of aircraft and spacecraft, sale of other motor vehicles, and maintenance and repair of motor vehicles all include a majority of out-of-scope activity for use in this division, so their weight has been reduced by a factor of 10 compared with other component measures.

The COICOP group 09.6 (Package holidays) has no UK-level data for the domestic concept, nor in either of the international tourism flows. It therefore also has no data in the national concept, although conceptually it could exist there. The reason for its absence is because in the UK National Accounts the spending is separated out between flights and hotels, so including it again would be double-counting.

COICOP 10: Education

For this division we have LCF data for the national concept, but there exists a potential problem in coverage of students in higher education, as the LCF does not include those living in communal establishments, such as halls of residence. The sample size achieved for this division is also rather small.

For the domestic concept, we have used ABS turnover for businesses classified to SIC 2007 codes: 85.1 (Pre-primary education); 85.2 (Primary education); 85.3 (Secondary education); 85.4 (Higher education); and 85.59 (Other education).

COICOP 11: Restaurants and hotels

For this division we have LCF data with a class-level breakdown for the national concept. For the domestic concept, we have used ABS turnover for businesses classified to SIC 2007 codes: 55 (Accommodation); and 56 (Food and beverage service activities).

COICOP 12: Miscellaneous goods and services

For this division we have good coverage for the national concept, with LCF data at the class-level for most components. For the domestic concept, we have ABS retail sales by commodity for personal care equipment and personal effects. For other components we have used the regional distribution of ABS turnover for businesses classified to SIC 2007 codes:

• 65.11 (Life insurance)
• 65.12 (Non-life insurance)
• 68.31 (Real estate agencies)
• 69.1 (Legal activities)
• 80.1 (Private security activities)
• 80.3 (Investigation activities)
• 87 (Residential care activities)
• 88 (Social work activities without accommodation)
• 96.02 (Hairdressing and other beauty treatment)
• 96.03 (Funeral and related activities)
• 96.09 (Other personal service activities)

We have no regional data on spending on prostitution. We therefore have used the same method we use to allocate prostitution in our regional gross value added measures, which is to use the regional distribution of adult population numbers. This means the regional allocation of prostitution merely reflects the overall population and contains no true information about regional variation in the use of prostitutes.

The COICOP group 12.6 (Financial services) presents some particular measurement issues. We have LCF data for this category, but it does not include financial intermediation services indirectly measured (FISIM), which represents the invisible charges made by financial institutions in the management of customers’ loans and deposits.

For the domestic concept, we have obtained data from the Bank of England that provide estimates of FISIM and bank and building society fees and commission income at the NUTS1 level of geography. For the national concept, we have estimated the regional distribution of FISIM by using data from our gross disposable household income (GDHI) measure on interest payments on mortgages and other loans, and interest received on savings and other investments to represent household loans and deposits respectively.

We have compiled provisional estimates for the NUTS1 countries and regions of the UK, for the years 2009 to 2016, which are consistent with the UK National Accounts, The Blue Book 2017 and regional gross disposable household income (GDHI) published in May 2018.

The full set of results, including its detailed commodity breakdown, can be found in the dataset published with this article, which you can download as a spreadsheet. Here we present a summary of the data showing the main items at a high level, to give a flavour of what is available in the full dataset. All the figures are compiled and presented in current market prices, which do not remove the effect of price inflation.

To begin, it is useful to say a little about what the data mean. The estimates of domestic expenditure relate to all spending that takes place in the region, but do not necessarily relate to the people who live there, since people can and do travel around the country, spending as they go. It therefore makes little sense calculating domestic expenditure on a per person basis.

National expenditure for a region does relate directly to the spending by the people who live there, so calculating national expenditure per person is useful, as it allows us to compare spending across regions of different size and population. Total national expenditure is also the value that is used in the calculation of the households’ saving ratio.

Where domestic expenditure is useful lies in its use in helping us to derive estimates of net spending flows between countries and regions of the UK. In this we also need to take into account spending by foreign visitors to the UK and spending by UK residents abroad.

Table 2 shows total national expenditure on all goods and services and national expenditure per head of population for the NUTS1 countries and regions of the UK in 2016, and the percentage growth in spending per person between 2015 and 2016. For comparison purposes, the overall rate of inflation for this period, as shown by the Consumer Prices Index (CPIH all items), was 1.0%.

London had the highest national expenditure per person in 2016, at £24,545, mainly driven by the higher housing costs in and around the capital. The lowest spending per person in 2016 was seen in the West Midlands, at £15,276.

In terms of growth in spending per person between 2015 and 2016, the North East had the greatest increase, at 8.1%, which is the highest annual growth seen in any region in any year since 2009 (the earliest year for which we have regional estimates). This growth was seen across a wide range of goods and services, with the strongest growth seen in the household goods and services, and clothing and footwear categories.

The lowest growth in spending per person between 2015 and 2016 was seen in Northern Ireland, at negative 0.4%, the only country or region to see a fall in spending per person in this period.

We can explore what the people in each country and region of the UK are spending their money on by looking at the broad categories of goods and services. Tables 3a and 3b present these figures on a per person basis for 2016, so we can compare across different areas on a consistent basis.

Here we can clearly see the much higher housing costs in London and the South East. We can also see that spending in the South East is generally high across a wide range of commodities, compared with most other parts of the UK.

Some curious results are visible, which may be caused by some of the weaknesses in the data sources used, but are still worthy of note in case they reveal an underlying truth. For example, spending on tobacco in Northern Ireland is conspicuously higher than in any other region of the UK. Spending on food and clothing in Yorkshire and The Humber are both notably lower than in other regions, while in London spending on recreation and culture is by far the lowest of all countries and regions. This last example may be a result of the higher housing costs in the capital, if people are being forced to spend less on voluntary things such as recreation because their essential costs are so much higher.

We have mentioned that having both domestic and national expenditure allows us to estimate the net household spending flows between countries and regions of the UK. Table 4 shows the derivation of these estimates, which are calculated as total domestic expenditure, less spending in the UK by foreign visitors, less total national expenditure, plus spending by UK residents abroad. Positive net spending figures imply that the region is a net exporter of goods and services to the rest of the UK, whereas negative figures imply the region is a net importer of goods and services from the rest of the UK.

We can see that, in 2016, only London and the West Midlands were net exporters of goods and services to the rest of the UK. It is possible that the way we have allocated financial services in domestic expenditure may have assigned more to London than should be the case. However, we would expect London to have by far the greatest share owing to the dominance of The City in financial affairs.

For the West Midlands, the two largest areas showing net exports in 2016 are in the supply of water and gas, and in out-patient services in healthcare. While the former appears sensible in the presence of large utility companies doing business across the UK, the latter seems unusual and may be a result of some of the measurement issues we have with these data. You are advised to exercise caution in the interpretation and use of these estimates.

Having compiled estimates of total household final consumption expenditure by the national concept, we can use these estimates to extend the regional household account from its current end point, gross disposable household income (GDHI), and derive the households’ saving ratio for the countries and regions of the UK. The saving ratio is the percentage of total available resources that is left after all spending has occurred (gross saving divided by total resources). Table 5 shows the stages in this derivation, using data for 2016. The transaction codes shown (B.6g to B.8g) correspond to those used in the UK National Accounts.

We can see that the households’ saving ratio varies considerably across the countries and regions of the UK, with saving in London and the West Midlands being the highest in 2016, at 14.5% and 12.8% respectively. The lowest levels of saving in 2016 were seen in the South West, at 1.5%, followed by Northern Ireland and Wales, at 2.5% and 2.6% respectively. These figures compare with a UK average saving ratio of 6.9%.

We have made good progress in finding appropriate data sources and solving some of the measurement issues that are peculiar to the compilation of regional statistics. However, there remain some issues that we have yet to resolve, or for which we have developed interim solutions but still need better long-term solutions.

Length of time series

One major issue is the variable length of time series available across our data sources. We have attempted to obtain data as far back as possible, in order that we might produce statistics for historic periods. Ideally, we would like to match the time series provided in our gross disposable household income (GDHI) publication, which contains data back to 1997. However, so far we have only been able to obtain full data for all sources back to 2009.

One of the problems faced in obtaining data for earlier periods is the change in industrial classification that took place around 2010, when we adopted the Standard Industrial Classification 2007: SIC 2007. Business survey data, including those from the Annual Business Survey (ABS), are only available back to 2009 on a consistent basis. Earlier data would require an industrial conversion that is both time-consuming and difficult to ensure that it produces accurate results. For our use of very finely detailed industries, this would be unlikely to provide reliable results.

Other data sources are available for variable lengths of time. The energy consumption data obtained from the Department for Business, Energy and Industrial Strategy (BEIS) are available back to 2004 or 2005. The data on parking revenue from local government are available back to 2003 for Wales and Scotland, 2004 for Northern Ireland, but only back to 2008 for England.

The Living Costs and Food Survey (LCF) data are available back to 2001 with consistent commodity definitions, but we have so far only obtained regional estimates back to 2009.

The unavoidable choice that we must make is whether to only publish statistics for which we have all, or most, of the necessary source data, or whether to use modelling techniques to project back to earlier periods. Clearly there would be a significant loss of quality in any such modelled estimates, but it may be that users would prefer to have some data, even if the quality is poor, than no data at all. This is another issue that we would like to hear about from people in the following consultation.

Other issues with data and methods

Some of the BEIS energy data are not available for the latest year until after our planned publication date for regional household expenditure. The data for liquid and solid fuels, and some of the data for Northern Ireland, are produced in September, which is too late for our summer publication. We propose to use the corresponding LCF data, which are available for the latest year, to estimate growth between the latest two years (using LCF for both), which will be used to project forward from the latest available BEIS energy consumption data. These provisional estimates will then be revised using the BEIS data in the following year’s publication.

Although many of our data sources are able to provide estimates for areas down to the local authority level, this is not the case for all sources. In particular, the data on actual and imputed rental from the Valuation Office Agency (VOA) and devolved administrations, and the data from the Bank of England on direct and indirect banking charges, are only available at the NUTS1 level of geography. We will need to use different data or methods to produce estimates for smaller areas. For rental, we already have an established methodology using dwelling stock and median house prices to estimate rental values, so we can use this again for household expenditure. For banking services, we will need to develop a modelling approach for smaller areas.

Another issue concerns the arbitrary weighting of turnover components used in combination to provide a good match to the commodities within certain COICOP classes, particularly those in the recreation and culture division. Since some of these components include a majority of activity that does not relate to households (such as aircraft maintenance), or relates to a wider scope (such as the use of motor vehicle maintenance to cover recreational vehicles, or the sale of other motor vehicles, which includes heavy goods vehicles), we have reduced their impact by a factor of ten. We have no true information on how much of the data actually relates to the item of interest, and it is possible that the factor is very inaccurate.

As mentioned earlier, the LCF data for education does not include students living in halls of residence, which may affect the quality of estimates for the national concept. So far, we have been unable to obtain a suitable regional data source to address this under-coverage.

The data used to allocate the international tourism flows (spending in the UK by foreign visitors and spending abroad by UK residents) to regions of the UK come from the International Passenger Survey (IPS). However, they do not include a commodity breakdown. We have a commodity breakdown for the UK as a whole, but for the regional allocation we have assumed the same composition of goods and services applies equally across all countries and regions.

Finally, we have no regional data for illegal activities (narcotics and prostitution) and have used the distribution of the adult population to allocate these activities across the UK. If we can identify a suitable data source that is reliable and consistent across UK countries, we will make use of it in our measures of both gross value added (GVA) and household expenditure. Until then it is important that users are aware that these estimates do not contain any real information about the regional variation in illegal activities, and no such inference can be derived from the data.

This article and its accompanying dataset is the first step towards a fully integrated measure of regional household expenditure. Its purpose is to show what is currently possible and explore the strengths and limitations of the data currently available. We will follow this publication with a public consultation, in which we will seek the views of users on what we have achieved here, and where they want us to focus our efforts in the future.

Ahead of that, it is worth taking a look at some of the things we expect to make a difference in the coming years.

Boosts to the Living Costs and Food (LCF) Survey

Recently both Scottish Government and the Northern Ireland Statistics and Research Agency (NISRA) have provided funding for a significant boost to the LCF sample in their respective countries. We expect to see an improvement in the quality and reliability of LCF estimates for Scotland and Northern Ireland as a result, and as the boosts have been designed to cover all areas, we also expect to see better estimates for smaller areas of these countries in the data for 2017 and beyond.

In addition to these country-specific boosts, we have used some of the funds from the Devolution Project to add some extra questions to the wider Survey on Living Conditions (SLC), which has a combined sample size of 12,000 households. While this will not provide the detail of spending that we get from the LCF, the additional information should enable better modelling and estimation of small area estimates across the whole UK. The extra questions have been included for two years beginning from April 2018.

Credit and debit card data

Since the enactment of the Digital Economy Act 2017, we have been engaged in the process of identifying and negotiating the acquisition of additional data from a variety of administrative and commercial sources. One avenue we have been exploring is the acquisition of anonymised data on credit and debit card purchases from the companies that issue the cards.

If successful this may provide useful additional data on purchases by households, with the location of the purchaser and the outlet used both being included at a level that protects the confidentiality of people’s personal information, enabling a large amount of extra data to be fed into both concepts’ measures. The main weaknesses we have identified so far lie in the categorisation of merchants to provide a good match to the commodities we want to measure, and the limitation that cards are not the only means used to make purchases. Nevertheless, if we can secure these data it should make a big improvement to the quality and reliability of our estimates, particularly for smaller areas.

This is not the only potential source of additional data that we are investigating, but in the short-term at least it appears to be the most promising.

Next year’s publication

This article and its dataset will be followed by a public consultation, in which we will gather views from a wide range of users on the data and methods we have used here, the various issues we have identified that need more work to fully resolve, and the provisional results we have produced.

We should stress again that these results are experimental and are subject to many potential sources of statistical error, for the reasons explained in this article. You are advised to use these data with caution.

Once we have gathered the views of users, we will firm up our plans for a more extensive dataset covering other geographic areas of the UK. We will examine the impact of sample changes on results and their sensitivity to some of the assumptions we have been forced to make, with a view to improving our methods and reducing the volatility of results, particularly for smaller areas. We will also look to take on any better data sources that become available to us.

Our current aim is to publish the first full dataset as Experimental Statistics in the summer of 2019, following our regular publication of regional gross disposable household income (GDHI). The development of these statistics will take several years to achieve fully reliable measures at all geographic levels. This is merely the first step on that journey.