Snap Inc.
Snap's My AI chatbot: UK regulator ends case without enforcement after Snap redoes its risk assessment
Snap launched its My AI chatbot, built on OpenAI technology, to Snapchat+ subscribers on 27 February 2023. The UK data protection regulator issued a provisional enforcement notice in October 2023 over the teen-risk assessment, then closed the case without enforcement in May 2024 after Snap submitted a revised assessment that it found compliant with the law.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
No grounds to issue an Enforcement Notice
Outcome of the ICO's provisional enforcement notice of 6 October 20231
Fifth
Version of Snap's data protection impact assessment that the ICO found compliant (22 November 2023)1
approximately 35.5%
Share of My AI's average daily UK users aged 13-17, 1 March to 18 April 20231
approximately 13%
Share of UK monthly active users of My AI aged 13-17 in April 20231
What it took
The problem
Snap added a generative AI chatbot, My AI, to Snapchat, an app with many users aged 13 to 17. UK law requires a company to assess the risks to people's data before it starts a new kind of processing. The assessment is called a data protection impact assessment. The UK Information Commissioner's Office (ICO) opened an investigation in June 2023, concerned that Snap had not adequately assessed the risks of My AI before launch.
What they built
My AI is a chatbot inside Snapchat where users ask questions by text, image or voice note. According to the ICO decision it is powered by OpenAI's GPT technology, which Snap uses through OpenAI's developer API. Snap launched it for paying Snapchat+ subscribers on 27 February 2023 and for all UK users on 19 April 2023. The chatbot received guidelines written by Snap, user attributes such as an under-18 age bucket and city-level location, and earlier queries. Snap added screening of replies, and for queries about certain sensitive topics it returned vetted help information. It also used an internal model to detect commercial intent, so that a contextual advertisement could be placed in a reply.
How they did it
The regulator's file shows the sequence. Snap's first impact assessment is dated 24 February 2023. The ICO met Snap on 20 April 2023 and later issued two formal information notices to obtain every version of the assessment. On 6 October 2023 it issued a preliminary enforcement notice. Its provisional view was that the assessment had not met the legal requirements and that Snap should have consulted the regulator first, because its first four assessments appeared to rate the risk to 13 to 17 year-olds as high. The draft order would have required Snap to stop processing UK users' personal data for My AI until a revised assessment was done.
Snap replied in writing and at a hearing on 14 December 2023. On 22 November 2023 it gave the ICO a revised, fifth assessment. Snap also told the ICO that the high-risk conclusion in the earlier drafts had been recorded in error, supported by a signed witness statement.
What went wrong
The ICO's provisional view was that Snap had launched without a compliant risk assessment, in particular on the risks to young users, on special-category data, and on how people would understand the use of their data. The ICO said Snap had invested considerable time and effort in the fifth assessment and had addressed the concerns in the notice.
Results
On 21 May 2024 the ICO decided the fifth assessment met the legal requirements and that Snap had not breached the duty to consult the regulator. It therefore found no grounds to issue an Enforcement Notice. The ICO described the case as a warning shot for industry about generative AI and data protection, and said it would continue to monitor My AI. It imposed no fine.
Lessons for your project
Finish the risk assessment before launch and treat it as the record a regulator will read. Snap's first assessment is dated 24 February 2023, just before the 27 February launch, and the regulator's provisional view was that the early versions fell short.
Keep risk ratings consistent across drafts. Earlier versions recorded a high residual risk for 13 to 17 year-olds, which would have required consulting the regulator. The ICO accepted that this was an error, on a signed witness statement.
Engage the regulator before launch. Snap asked for a meeting in March 2023, but the first meeting was on 20 April 2023, almost two months after the subscriber launch and a day after the wider rollout.
A rewritten assessment can end a case. The ICO credited more detailed descriptions of the processing, a specific look at the risks to young users and clear mitigations.
Plan for scrutiny of younger users when a general chatbot sits inside a social app. The regulator focused on risks to under-18s, special-category data and advertising.
Caveats
- This is a regulator's process, not an enforcement result. The ICO's October 2023 notice was provisional, and the May 2024 decision found no grounds to issue an Enforcement Notice. No fine was imposed.
- The decision covers only Snap's compliance with the UK GDPR rules on impact assessments and prior consultation (Articles 35 and 36), as of 22 November 2023. The ICO says it made no findings on My AI's wider data processing. It also said it would keep monitoring.
- Absolute user numbers are redacted in the published decision; only some percentages are legible. The age shares are Snap's figures to the ICO, not independently audited.
- The ICO's decision names OpenAI's GPT technology, used through OpenAI's developer API, but not the model version. Snap's advertising for My AI used Microsoft as a processor, per the decision.
- The ICO's earlier press release on the October 2023 notice could not be opened, so that step is taken from the decision itself.
Sources
- Decision of the Information Commissioner: Snap Inc. and Snap Group Limited, 'My AI' (21 May 2024)
UK Information Commissioner's Office · 2024-05-21 · Filing · ico.org.uk
- We warn organisations must not ignore data protection risks as we conclude Snap 'My AI' chatbot investigation
UK Information Commissioner's Office · 2024-05-21 · Press release · ico.org.uk
Summarized in our own words from the sources above. Spotted an error? See corrections.