Austria's job agency AMS launches a ChatGPT career bot and gets gender-bias criticism

The problem

Austria's public employment service, AMS, wanted to give young people and jobseekers a faster way to explore career paths. It also hoped to relieve its human advisers.

What they built

AMS presented Berufsinfomat, a chatbot that is built on ChatGPT and linked to AMS's own occupation encyclopedia. It answers questions about careers and cites the AMS entries with links. An outside AI company that sells ready-made bots built it. Futurezone names that company as Goodguys.ai, and says the software was adapted only lightly from a standard product.

How they did it

The bot combines ChatGPT's general knowledge with AMS occupation content. AMS's chief Johannes Kopf said it was tested for months with advisers and users before launch. A hidden instruction told the model not to distinguish between men and women. AMS states a cost of about 300,000 euros, and shows a disclaimer that error-free answers are not guaranteed.

Results

The launch was overshadowed by testing from the public. Netzpolitik reports that for a man with excellent school grades the bot suggested IT and retail, and for a woman it suggested gender studies or philosophy. Futurezone reports similar patterns, with women pointed to care, beauty and fashion and men to software and warehouse work. A tester also got invented answers for a made-up job. AMS said it had trained the system to make as little distinction as possible and that work continued. Outside developers also reported code and security flaws.

What went wrong

The bias appeared quickly, in the hands of ordinary users. Kopf's posts, as read by netzpolitik, suggested AMS knowingly accepted such weaknesses in return for learning. A mathematician quoted by netzpolitik said the problem cannot easily be corrected after the fact. Privacy was a second issue: user inputs go to OpenAI, and AMS put the responsibility on users with warnings.