Department of Families, Fairness and Housing (Victoria, Australia)
Victoria's child protection department ordered to block AI tools after a worker used ChatGPT in a court report
A Child Protection worker at Victoria's Department of Families, Fairness and Housing used ChatGPT, without approval, to help draft a report for the Children's Court. The state privacy regulator, OVIC, found the department breached two privacy principles and ordered it to block generative AI text tools for Child Protection staff; the department accepted the findings.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
nearly 900
Department employees who accessed the ChatGPT website, July to December 2023 (almost 13 per cent of its workforce)2
100
Child protection cases with indicators that ChatGPT may have been used, found in the department's internal review of one work unit over a one year period (the review covered 796 cases)2
IPP 3.1 and 4.1
Privacy principles OVIC found the department contravened2
by 5 November 2024 until 5 November 2026
Period the compliance notice requires blocking of generative AI text tools for Child Protection staff2
What it took
Cost, team size and timeline were not disclosed in the sources.
The problem
In December 2023 the Department of Families, Fairness and Housing told the Office of the Victorian Information Commissioner (OVIC) that a Child Protection worker had used ChatGPT when drafting a Protection Application Report. That report went to the Children's Court in a case about a young child. The department said the worker had breached its acceptable-use policy. Generative AI tools write fluent text that can be wrong, and anything typed into them leaves the organisation.
What they built
Nothing was built. This was unsanctioned staff use of a public tool, ChatGPT from OpenAI, on department devices. The department pointed to general policies, including an acceptable-use policy and training on privacy and security. OVIC found these gave staff no clear guidance on ChatGPT or generative AI.
How they did it
OVIC began an investigation to decide whether to issue a compliance notice. It looked at whether the department took reasonable steps to keep personal information accurate and to protect it from misuse. After the incident the department reviewed the cases of the worker's wider work unit over a year, and checked audit logs for July to December 2023. In April 2024 its Chief Information Officer asked the nearly 900 employees who visited ChatGPT for examples of safe use. Only ten replied.
What went wrong
The worker admitted using ChatGPT for the report and for other work to save time, but denied entering personal information, saying names were removed. Neither the department nor OVIC could check what was typed, because organisations cannot see staff inputs to ChatGPT. OVIC found, on the balance of probabilities, that personal information including names was entered. It cited signs of ChatGPT throughout the report, frequent use of individuals' names in those sections, and colleagues' accounts that the worker had typed client names into the tool when demonstrating it.
OVIC also found the report contained inaccurate personal information that downplayed risks to the child, and that the entered information reached OpenAI, an overseas company, outside the department's control. The Deputy Commissioner found the department's controls were insufficient and that it breached Information Privacy Principles 3.1 and 4.1. Neither the department nor OVIC saw a change in decisions about the child.
The department's review of 796 cases from the worker's wider work unit found 100 with indicators of possible ChatGPT use. Department reviewers judged that none of these had the same effect on risk assessment as the court report, and OVIC, after seeing nine samples, shared that view. OVIC added that the samples suggested more documents than the one report had been drafted with ChatGPT, and that reviewers often could not tell ChatGPT use from wider practice problems in the work unit.
Results
OVIC issued a compliance notice with six specified actions. These required a direction to Child Protection staff not to use web-based or API-based GenAI text tools for official duties, IP or DNS blocking of named tools, scanning for similar new tools, and controls to stop Child Protection staff using Microsoft 365 Copilot. The blocking was due by 5 November 2024 and must be kept until 5 November 2026. The department also had to report to OVIC on how well the blocking works, on four dates between March 2025 and September 2026.
The department's Secretary accepted the findings and committed to making all reasonable efforts to meet the specified actions on time. The department described the matter as an isolated incident and said GenAI use is not prolific, a view OVIC did not accept in full (see caveats). The department also noted there is a process to seek OVIC's advice on removing the requirements, and OVIC's report says the department may apply to drop individual blocking actions if it shows evidence of better controls.
Lessons for your project
Do not rely on a general acceptable-use policy. OVIC found the department gave no clear training or guidance on ChatGPT, so staff could not tell whether or how to use it.
Check what staff already use. The department's audit logs showed nearly 900 employees visited the ChatGPT site, but they showed only access, not what was typed.
Block tools where the data is highly sensitive. OVIC held that policy and guidance alone could not manage the risk in child protection, and ordered technical blocking.
Plan how staff report AI use. Only ten of the nearly 900 employees answered the department's April 2024 request for examples of GenAI use.
Caveats
- This is unsanctioned use by staff, not a deployment the department chose. It qualifies as a failure case because a regulator documented the use and the department's response.
- The department's account differs from OVIC on some points. It says this was an isolated incident, that GenAI use is not prolific, that the report could not confirm personal information was entered by the practitioner (who has left), and that no decision about the child changed.
- OVIC agrees no decision changed and that inputs could not be verified, but found on the balance of probabilities that names and other personal information were entered, despite the worker's denial. It rejected the department's claim that no sensitive-work use was found, and said its samples suggested more documents were drafted with ChatGPT.
- OVIC says the department's figures are indicators only. Reviewers often could not tell if poor content came from ChatGPT or from wider practice problems in the work unit.
- The 100 cases are those with indicators that ChatGPT may have been used, out of 796 reviewed. The 900 employees only accessed the website on a department device. Neither figure proves what was entered.
- The deployed date shows 2023, the year of the use found. The incident was reported to OVIC in December 2023.
Sources
- Investigation into the use of ChatGPT by a Child Protection worker
Office of the Victorian Information Commissioner · 2024-09-24 · Filing · ovic.vic.gov.au
- Investigation into the use of ChatGPT by a Child Protection worker (investigation report)
Office of the Victorian Information Commissioner · 2024-09-24 · Filing · ovic.vic.gov.au
- Victorian agency breaches privacy through use of ChatGPT
Information & Data Manager · 2024-09-26 · News · idm.net.au
Summarized in our own words from the sources above. Spotted an error? See corrections.