The Washington Post
Washington Post launches AI podcasts after tests found most scripts failed
In December 2025 The Washington Post released 'Your Personal Podcast', an AI tool that makes personalized audio briefings, as a beta. Semafor reported that internal tests found 68% to 84% of generated scripts failed the newsroom's standards across three rounds.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
between 68% and 84%
Share of generated scripts that failed the Post's pass/fail standards test, range across three rounds of internal testing1
What it took
Cost, team size and timeline were not disclosed in the sources.
The problem
News publishers want to offer personalized products. The Washington Post wanted a tool that turns its reporting into a custom podcast for each reader. A news brand's value rests on trust, so accuracy matters more than speed to market.
What they built
"Your Personal Podcast" is an AI tool that generates personalized audio briefings from the Post's journalism. The Post's technology and product heads announced it to staff as part of its push into premium experiences, customer choice and AI products. The Post labels it a beta.
How they did it
Before release, the Post ran internal tests in which testers rated each generated script as pass or fail against the newsroom's publishing standards. Testers were told to fail a script when in doubt. The Post's review team still recommended moving ahead and continuing to fix the remaining issues with the newsroom, while marking the tool as a work in progress.
What went wrong
Semafor reported that between 68% and 84% of scripts failed across three rounds of testing. The internal review concluded that further small prompt changes were unlikely to help without adding risk. Staff described mistakes from pronunciation slips to fabricated or misattributed quotes, and said the tool sometimes added commentary, such as treating a source's quotes as the paper's own position. Some editors objected strongly. A Post spokesperson said the product was clearly marked as a beta.
Results
The product went live as a beta despite the test results. The case is a record of a release decision, not of business results. The source gives no usage, subscriber or cost figures.
Lessons for your project
Set a launch threshold before testing, and honor it. The Post's own review said small prompt changes were unlikely to help, yet the product shipped.
Involve editorial and standards staff early. Newsroom leaders objected after learning of the errors, and one editor wrote that the tool should be pulled immediately.
Labeling a product 'beta' does not remove the risk when it publishes fabricated or misattributed quotes under your brand.
Use a strict test rubric. The Post's testers were told to fail scripts when in doubt, which surfaced issues, but it also means the failure rate reflects a cautious standard.
Caveats
- The only source is one Semafor article, based on an internal test readout and staff accounts shared with the outlet. The Post's own test results were not published.
- Testers were told to fail scripts when in doubt, as a precaution, so the 68% to 84% rate reflects a strict standard.
- The article does not name the AI model or vendor.
- The article says the product was announced on a Wednesday before its December 14, 2025 date, and does not print the launch date. The deployed month shown is inferred from that.
- The case is about the state at the time of the article. Later changes, such as fixes or withdrawal of the product, were not checked.
Sources
- 'Iterate through': Why The Washington Post launched an error-ridden AI product
Semafor · 2025-12-14 · News · semafor.com
Summarized in our own words from the sources above. Spotted an error? See corrections.