This transcript was produced using AI and has been adjusted for clarity.
James Benford, Director General for Surveys and Economic Statistics: I was just going to talk you through basically the document that's just been published and then starting off with what's improved and what we've delivered since the last quarterly update. And first thing to say is we've had the latest data following the implementation of the last set of agreed design changes on the TLFS, including data rotation. And that's been really encouraging, actually. And we can see an improvement in responses across particularly the later waves of the survey. And that sets the scene for our first readiness assessment, which will, we're going through that at the moment and in August will set out the path towards transitioning to the TLFS as our main headline measure for labour market statistics in 2027.
So encouraging progress there. The Statistical Business Register, or SBR, has moved to a stronger iterative delivery model and early benefits are actually already being deployed through enhanced search capabilities within the ONS on that register. The new approach, the iterative approach, releases value to users earlier and is allowing us to gather feedback iteratively. In national accounts, the IMF have accredited UK statistics as meeting their highest tier of standards, so-called special data dissemination standard plus.
The OSR has re-accredited business enterprise research and development statistics as official statistics, again, reflecting the improvements we made to those statistics last year in the Blue Book, improving their coverage. And we've published a broad package of measures to set out how we assess seasonality and GDP.
And [we have] completed the move to the remaining Annual Survey of Hours and Earnings to a digital first collection, reducing operational costs and risk in that survey and actually vastly speeding up data processing. So, a lot's been achieved.
Those achievements build on longer term progress, which has seen the Labour Force Survey response rates go back to the response levels, pre-covid levels, reinstatement of producer price statistics, grocery scanner data brought into CPI inflation, and the use of our in-house ClassifyAI tool, halving processing time for occupation coding in a number of our surveys.
The release this time includes a set of key performance indicators. There's more detail in the document, but I've pulled up the headlines on this slide. As our previous updates have told you, we've been putting in more resources into core surveys and statistics - 214 extra people over the past year in economic statistics production, 71 in business surveys, 138 in social surveys, which is more than accounted for by expansion in the field for us. We're seeing the benefits of that.
So, there's been fewer errors since the last update just around half - is a little over half - the amount of total errors that we that we've made relative to the last update. And we've made no major errors to any of our economic statistics so far this year. We're seeing, as I mentioned, increased achieved survey sample sizes. That is most prominently in the Labour Force Survey, but it's not just that. We're seeing expansion in the achieved sample size for the Living Costs and Food Survey and lots of encouraging signs across our business surveys, particularly the labour market surveys where we've been focusing our efforts. Improved performance on the business surveys reflects progress moving them online and expansion of our large case and account management units. Also, [it] is facilitating the fact that the business services are mandatory.
We've got ongoing work as well on the household surveys to explore other countries' experience with mandating household surveys and to understand the potential costs and benefits there.
Of course, you know, it's not it's not all been good news on performance and three weeks ago, I took the early step of alerting people publicly to an issue that we had with the Labour Force Survey, where because of an over allocation - essentially an operational error from the from telephone interviews from the LFS to the TLFS - we're basically going to have a temporary hit to the achieved sample size for the LFS in the coming months. It's relatively small, it's manageable. The issue is now being corrected and we're well in to conducting a lessons learned exercise now. I'm confident our methods can manage the impact on the quality of our statistics and we'll publish our estimates of the impact in our next labour market release. Nevertheless, I thought it was important to share the issue as soon as practical in line of our commitments to transparency.
It's clear to us at least that a number of our users are recognising the improvements in the quality of our surveys and income statistics. The Resolution Foundation and NIESR have in recent weeks actually publicly recognised the progress we're making on the quality of our outputs and also our commitment to transparency improvements in our communications. And both the Bank of England and the Office for Budgetary Responsibility have also recognised improvements in labour market statistics and our approach to communications too. I'm expecting, and we'll let you have it as they release it in the coming five minutes or so, a statement from the OSR this morning off the back of this update, acknowledging the improvements that have been made and adjusting their regulatory stance in light of that. So, we will share that with you when it comes out.
Interestingly, last month I was invited to give an inaugural keynote address at the OECD's Committee for Statistics and Statistical Policy to share basically how the UK is responding to the Deveraux Review last year and the challenges that were kind of highlighted by that review. Discussion was pretty striking, I found, basically for how widely shared the challenges faced by the ONS are, namely increasing demands for more, more statistical outputs, pressure to keep pace for technological change, but the imperative also to maintain quality. I came away pretty encouraged that the path that we're taking at the ONS is one that others, including the OECD's Chief Statistician recognises as the right one.
You of course want to make your own judgments on how we're doing and you know we still do have much further to go and that's why I thought it would be a good idea to hold this Q&A session press conference today and you know welcome your feedback as well after the session if this kind of thing is useful.
We are under no illusion that there's a lot more to do and I've laid out the big-ticket items we have in front of us for the year ahead. Of course, the top priority is transitioning to the Transformed Labour Force Survey next year. We will tell you in detail what our plans are on that in August after we complete the readiness assessment.
Close behind that is the Statistical Business Register, adopting that across our surveys and integrating it in our systems. That is a crucial foundation to be able to update, to take on board the latest international standards. There's two really important pieces of methods work that we'll do actually this calendar year.
So, in the coming Blue Book, we are integrating a new survey. It's been running for a couple of years, but it's been brought into GDP for the first time, the Annual Surveys on Goods and Services, and that will give us a much clearer picture on goods production in the service sector, and it's going to greatly enhance our understanding of that sector. So we'll publish an article on that in August. And so, watch out for that. It's an important methods change.
Another one, and this is something that's had some commentary, actually, and reasonably so, it has become more challenging to measure productivity in the economy, not least because of some of the historic issues affecting the quality of the LFS, which still do affect the year-on-year growth rate of employment in that survey, which feeds through to productivity. We put out a note on that recently, a blog by Richard Heys, guiding people towards an alternative measure of productivity based on the HMRC data. But what we're working towards is a component-based productivity measure. It's in line with best practice and the early signs are it's closer to that RTI measure from HMRC. And that's why we put out the blog indicating that. We'll be publishing a methods article on that in September, and then we're aiming to progress towards that as our main headline estimate for productivity by November. So, two really important methods changes there.
We continue of course to focus on quality, where we're working with the Office for Statistics Regulation to re-accredit statistics which have lost their accreditation. We'll tell you in the August updates on the TLFS our plans to re-accredit labour market statistics and we have quite advanced plans working with the OSR to get producer price statistics and trade statistics ready for reaccreditation, which are mainly about making sure we have the right controls to manage risks in legacy systems.
And then looking ahead, and we're beginning to lay out the path here, there's two really big important things that we have to adapt for to make sure our economic statistics remain relevant. Last year, new standards for national accounts, so-called SNA 25, and balance and payments, so-called BPM 7, were agreed, and we now have to bring those into our statistics by 2030. And there's a sequence of changes that has to happen there. It includes, for example, better measurement of cloud computing and data as an asset as well. But, and this is a topic of live debate across national statistics offices, I don't think those new international standards fully account for the form, pace and the scale of adoption of artificial intelligence that we're now seeing. So we are launching work in parallel to improve our understanding of artificial intelligence. Broadly, we know from our surveys, 30% of companies are using AI in their work, over 50% for large companies. So, this is being adopted quickly. There are a number of areas where this touches national accounts.
And we're working towards, and we'll publish an article in September here, a so-called thematic account for AI that we're built towards iteratively over the year. And by the end of the year, we're going to work towards having a fully-fledged development plan to incorporate and adjust our statistics for AI, really important for maintaining the relevance of the ONS statistics.
So, you know, we are still constrained on how much we can do. When we're double running an LFS and a TLFS, there's only so much we can do. And we're constrained by legacy systems and a complex change agenda and then the need for a specialist workforce that comes with what we're trying to do.
And so we have held back some work, mainly to move off legacy systems in a waiting room, and we cannot transform the wider system of household surveys beyond labour force surveys until the TLFS transition is complete. The documents published today set that out. I mentioned already the LFS interviewer allocation issue does show the complexity of running two massive surveys in parallel. And when we're learning from that, that issue. But yeah, we are running with risk still.
Just to just to summarise overall, we've now integrated what were two separate plans, the survey plan and economic statistics plan into one integrated plan.
We are seeing further tangible quality improvements, and that's showing up in a new system of key performance indicators. Big year ahead, and it's, you know, perhaps the biggest ticket item is landing the TFS, and we're 100% focused on delivering that for next year, and we'll have more to say on that in August. Close behind the Statistical Business Register. Important methods changes coming in the Blue Book with that new survey of goods and services and the new component-based approach for productivity. And we're standing up work on AI as well as to find the path to implement the new international standards. That's basically what's in the document. There's more detail in the document. You maybe have read it as I talk through. Happy to take any questions on anything you like.
Q: Hey, just a very quick one, really, James. And obviously, you know, as you know, there's a lot of work been going on with some of the big questions. One of the silly little things that not worries me, that seems to be a bit of a problem, though, is almost like the death by a thousand cuts syndrome, that there's lots of silly little mistakes that keep coming into the data over a period of time. And over the last year or so, a lot of them have not really been ONS issues, but there have been issues on data that's fed in from the other ministries. How, is there any way you can put something in place to stop that, or is that just something that's going to be inherent to the system?
James Benford: So, it's a great, great question. I mean, before going to the beyond ONS issues, you know, we still do make mistakes. I mean, we're transparent on those. They have been minor, but you know, looking behind them, they largely relate to manual processes. And so, moving off legacy systems is going to help us kind of reduce the mistakes that we're making. And the OSR is also helping us think about the controls we need to have in place whilst we're still on those legacy systems running manual process.
On the beyond the ONS issues, I mean, you're right, there were at least four kind of fairly major errors last year that had their roots in the data that was supplied to us, HMRC, but also the VED on trade and public sector finance, and also the vehicle excise duty, error and CPI. We have, I did it jointly with Ed Humpherson (OSR), held a cross-government workshop, basically, with statisticians across government departments, but also importantly, the data professionals, running the admin systems. And we didn't do this as a way to kind of, you know, kind of share, kind of move blame around or anything like that. It was very much a collaborative exercise. And what we're trying to do is, you know, build one team on that end-to-end from the admin system to the statistic.
And it's quite clear to us from the workshop that there's more to do there. So quite often, when changes are made to administrative systems, you know, systems that kind of administer collection of tax data, for example, tax receipts, then people can make changes to those without being fully aware of the downstream impact on statistics. So, there's a mapping exercise that needs to be done and we will do that. And there's also a point around change control and having statisticians at the table when changes to those systems will be made. And we're looking at looking at that too.
The other, I mean, this is an important role for the ONS actually, that the ONS sits across many different data sets and, you know, has a key role drawing people together across the system too. And so, and this has to be a collaborative exercise, but making sure the right checks and balances are there.
So do our data sources look consistent with each other, and can we quickly raise a flag if that doesn't happen is important too. So, there's a lot that we can do and we'll be bringing actually all of that work to the UK Statistics Authority Board probably in September now.
And yeah, I think it's an area where we have been trying to show leadership across the system. I should just also say that there's a really important review that's ongoing by HMRC into the quality of their statistics. I've seen a draft of that. It's been a very thorough exercise. And yeah, I think it should be finalised shortly. And it's great to see that HMRC have done that in response to some of the errors that happened last year on both tax and trade. So yeah, we're working through that. I think the system's getting stronger.
Q: Just a couple of things. Just on the August readiness assessment, if all was kind of fine and dandy, is that the moment when you'll, you know, say we plan to launch in X month? or something like that? And also, you mentioned that the responses, response rates in the later waves of the TLFS were getting better. I don't know, I can't see any kind of firm numbers in today's release. Is that something you're holding back for now? In terms of what those response rates look like?
James Benford: We're doing the readiness assessment now and that's a process that is quite involved because it involves our users and we've got a stakeholder panel set up to do that. And it goes, you know, to the ExCo at the ONS and then it goes to the UKSA board. And then we publish. So, we're going through that and it wouldn't be right for me to kind of speak ahead of that process in terms of the detail. We will set out in August, though, kind of, you know, an indicative timeline and proceed from here on the TLFS. When we get to that point, I think you'll see that there's quite a number of steps on the road and you know, and it's a complex journey, but we will set out the timeline.
And it will be such that there will be, and you know, I won't say when this will be yet, but there'll be a final go, no-go type decision in 2027 on that timeline. So, you know, there will be a timeline which is indicative that, you know, will be driven by how the data pans out and we'll only go if we're ready to go. The question on TLFS data, we'll see what we can put out with the August update, decided not to put a lot of detail in this release to leave room to do it there. But, as I said, basically the data rotation, and we did change the incentives too, appears to have had the desired effect and the response rates in later waves have risen and have pretty much risen in line with what we've been targeting operationally. So that's really encouraging.
Q: Thanks for that. Just on that last mention, could you say what the incentives on the TLFS are at the moment? And then it would be good to hear a little bit more about how you're thinking on mandating or not mandating responses is going? How are you thinking about that? And are you going to at some point sort of put out formal recommendations one way or the other?
James Benford: Yeah. So, TLFS, we went from £10 to £20 in terms of the incentive. On mandation, this is a few things to say on this. It's, you know, it's a piece of work that's still to run. It's of course jointly with the Cabinet Office, because ultimately it would be for the government to make any decisions on whether to mandate or not. And it's clear from looking at other countries, and there's a chart actually that was shared at the OECD meeting that I mentioned, and to ESCoE recently from Statistics Canada, where they show the household survey response rates and the LFS response rate is, I think it's 78%, but all the other household surveys are around 25 to 30%, right? So, you can see that impact. But there's a few things to say as well on this. It's, you know, it's obviously a social construct, whether this is accepted that a survey is mandated and we have that for the census. So, there is a precedent in the UK. I do know examples of other countries where mandation can actually have kind of perverse results where people essentially fill the survey in but spoil it and don't complete it properly because they don't really want to fill it in. And that kind of gets to a deeper point, which is the real thing here is building ease of completing a survey, trust in what we'll do with those responses, and then shared understanding of the value of participating in the survey. I mean, that's the thing that ultimately is going to give us better quality data, and we can work on all of that, but without kind of going to mandation. So, yeah, I mean, I'd expect that, you know, we kind of reach a view and it is, as I described it, of the costs and benefits of mandation as a new national statistician joins potentially, you know, be something we'd involve the board in too. But ultimately, this is a decision for the government and for politicians because it's a change in the relationship between the state and citizens and so it's right it is a decision for them. And I'm focused on building a TLFS that works without mandation. So that's what we're working towards right now.
Q: Thanks. Can I just ask one other question? I just saw in your document you were talking about work to get a clearer view of the costs of delivering surveys and stats to sort of link the financial information more closely to the activities. Could you just tell us a little bit more about that, what that means?
James Benford: Yeah, it's something that the OSR asks us to do, and what they would like us to do is to publish basically for different sets of statistics, an indication of the cost to produce. It's not that straightforward to do because we use so much data across our statistics and also one piece of data was used in many places. We are going to try and do that and sometimes called activity-based costing. But yeah, it's taking a bit of time to do that and something we're working to do for the next update. I believe the reason they thought it would be useful to do is it could generate a more informed debate on, you know, the merits of continuing given statistics or not.