City of Yokosuka
Yokosuka City trials ChatGPT with about 3,800 staff, then moves to full adoption
In April 2023 the Japanese city of Yokosuka gave ChatGPT to every staff member who uses its internal chat tool, about 3,800 people. After 41 days 1,913 of 3,828 staff, about half, had used it, and 76.0% of final-survey respondents wanted to keep using it. The city also found accuracy complaints and announced it would move to full adoption.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
1,913
Staff with a ChatGPT chat room by 31 May 2023, out of 3,828 staff, over the first 41 days of the trial1
25,897
Messages sent to the ChatGPT bot in the first 41 days of the trial1
76.0%
Final-survey respondents (n=433) who said they want to keep using ChatGPT1
at least about 10 minutes
City's estimate of working time saved per day, as reported by Government Transformation2
What it took
The problem
Yokosuka, a city in Japan, wanted to know whether a general AI chatbot could cut the time spent on drafting, summarising and proofreading documents, without exposing personal data.
What they built
From 20 April 2023 the city made ChatGPT available to every employee who can use its internal chat tool, LoGo Chat, about 3,800 people. Staff talked to a ChatGPT bot inside the tool they already used. Personal information was not to be entered.
How they did it
The trial was run as an experiment with two staff surveys. The first ran from 27 April to 2 May and the second from 23 to 29 May. Between them the city sent out two issues of a staff newsletter with tips on using ChatGPT well. The questions in both surveys were the same, so the city could compare results. The city also interviewed users about how they saved time. The bot used ChatGPT-3.5-turbo.
Results
By 31 May, 1,913 of 3,828 staff, 50%, had a chat room with the bot, and they had sent 25,897 messages. In the final survey, the city reports that over 80% said ChatGPT would raise their work efficiency, and 76.0% wanted to keep using it. Separately, the city calculated a rough estimate of about 22,700 working hours a year for document drafting alone, assuming 10 minutes saved a day for the 29.3% of users who draft documents. Government Transformation reported the city's estimate as at least about 10 minutes a day saved.
What went wrong
Staff were not all happy. Many used ChatGPT for web-style search, which the city said suits it poorly, and the city reports that nearly half of survey respondents were dissatisfied with answer accuracy. The city responded with training and a plan to move to GPT-4.
All reported figures
| Measure | Figure | Citation |
|---|---|---|
| Staff with a ChatGPT chat room by 31 May 2023, out of 3,828 staff, over the first 41 days of the trial | 1,9131 | |
| Messages sent to the ChatGPT bot in the first 41 days of the trial | 25,8971 | |
| Final-survey respondents (n=433) who said they want to keep using ChatGPT | 76.0%1 | |
| City's estimate of working time saved per day, as reported by Government Transformation | at least about 10 minutes2 | |
| City's rough estimate (an assumption-based calculation, not a measured saving) of working hours saved a year on document drafting, from 1,913 users, 29.3% using it for drafting, 10 minutes saved a day and 243 working days | 22,700 hours1 |
Lessons for your project
Run a short, measured trial before committing. Yokosuka ran two staff surveys six weeks apart and used the first to fix problems in the second.
Teach people how to ask. The first survey showed many staff used ChatGPT for web-style searching, a weak use. A staff newsletter with tips reduced that share.
Put the tool where staff already work. Yokosuka connected ChatGPT to the chat tool every employee already used.
Set data rules on day one. The city did not allow personal information to be entered.
Plan for accuracy complaints. Staff dissatisfaction with answers stayed high, so the city planned a stronger model (GPT-4) and training.
Caveats
- The figures come from the city itself. The 22,700 hours is a rough estimate built from the city's own assumptions, not a measured saving; it multiplies users by an assumed 10 minutes a day.
- Survey response rates were low: 693 staff in the first survey (about 18%) and 433 in the final one (about 11%). Results may not represent all staff.
- Per the city report, more than 40% of final-survey respondents saw some inappropriate answers and nearly half were unhappy with answer accuracy. The city planned to move from ChatGPT-3.5-turbo to GPT-4.
- The city's report says full adoption, an AI strategy adviser and training were planned. Whether these were delivered is not covered by these sources.
Sources
- ChatGPT活用実証結果報告
City of Yokosuka, Digital Government Promotion Division · 2023-06-05 · Press release · city.yokosuka.kanagawa.jp
- Japan to use ChatGPT as a tool in city government
Government Transformation · 2023-06-27 · News · government-transformation.com
Summarized in our own words from the sources above. Spotted an error? See corrections.