HM Revenue and Customs
HMRC gives 3,000 randomly chosen staff Microsoft 365 Copilot and finds self-reported savings of around 60 minutes a week
HMRC ran a four-month trial of Microsoft 365 Copilot in which 3,000 licences went to randomly chosen staff, with 500 more for volunteers. Survey respondents reported saving 2-3% of their working week. HMRC treats the figure as a baseline estimate and not a precise measure.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
around 60 minutes per week
Same self-reported saving expressed per week for a full-time equivalent working pattern1
83%
Staff allocated a licence who made use of it during the trial period1
What it took
Cost, team size and timeline were not disclosed in the sources.
The problem
HMRC, the UK tax authority, wanted to know how much time an AI assistant could save its office staff in drafting, searching for documents and working in Teams. It also needed to know how many people would actually use one, and what held others back.
What they built
HMRC trialled Microsoft 365 Copilot, an assistant built into Word, Outlook and Teams that uses a large language model, an AI system that reads and writes text. The trial had three phases. Two early phases in mid-2024 involved 300 self-selected staff. Phase III was the most comprehensive.
How they did it
Phase III ran from September to December 2024. HMRC randomly allocated 3,000 licences to employees in roles that mostly use Microsoft Office, in proportion to business group sizes. A further 500 licences went to volunteers, including people who needed them as a reasonable adjustment. Evidence came from a survey with 1,364 responses, usage data from a Microsoft dashboard, 35 people in focus groups and 49 in a task exercise.
Results
Survey participants reported saving 2-3% of a working week, around 60 minutes for someone working full time. In total, 83% of staff given a licence used it, and satisfaction averaged 7.1 out of 10. Also, 61% would be disappointed to lose their licence. Drafting in Word (53%) and collaborating in Teams (45%) were the most used functions. The report says its estimates are more cautious than the Government Digital Service figure, which came from a different design.
What to copy
Randomise who gets the tool, discount the survey answers, and publish the method with the result.
All reported figures
| Measure | Figure | Citation |
|---|---|---|
| Time saved, as self-reported by trial participants, share of their working week | 2-3%1 | |
| Same self-reported saving expressed per week for a full-time equivalent working pattern | around 60 minutes per week1 | |
| Staff allocated a licence who made use of it during the trial period | 83%1 | |
| Average user satisfaction score | 7.1 out of 101 | |
| Users who said they would be disappointed to lose their licence | 61%1 |
Lessons for your project
Allocate licences at random if you want a fair read. HMRC drew 3,000 licences at random across business groups, which reduces the bias that comes from giving tools only to enthusiasts.
Discount self-reported savings. HMRC reduced self-reported time-saving estimates by about 20% because not every licence holder used Copilot and respondents were more likely to be active users, and called the result a baseline.
Mix several measures. HMRC combined a survey, usage data, focus groups and a task exercise instead of relying on one.
Set data-handling rules and train people. The top reason for non-use was security and privacy concern, and users asked for practical, tailored training.
Caveats
- HMRC evaluated its own trial. The savings are self-reported, and the report says they should be treated as a baseline estimate and not a precise measure.
- The report says self-reported time-saving estimates were reduced by approximately 20%, because not all licence holders used Copilot and respondents were more likely to be active users. The page does not say whether the 2-3% figure is before or after that reduction.
- The survey had 1,364 responses, 45% of the randomly allocated licence holders, so non-response may bias results.
- The report is a summary of one phase of an ongoing evaluation. Participants could use Copilot only on material below the Official Sensitive classification.
- The page also cites a roughly 50 million pound annual benefit estimate for wider licences. That is a projection, not a measured result, and is not used here.
- The sources name the product, Microsoft 365 Copilot, but not the underlying language model. Phase III is HMRC's own trial; HMRC also took part in the separate cross-government trial run by the Government Digital Service.
- Phase III ran from September to December 2024; the report was published in July 2026.
Sources
- Evaluating the Impact of Microsoft Copilot in HMRC
GOV.UK (HM Revenue and Customs) · 2026-07-09 · Research paper · gov.uk
Summarized in our own words from the sources above. Spotted an error? See corrections.