Zalando
Zalando rebuilds its fashion Assistant on GPT-4o mini and takes it to 25 markets
Zalando, a European online fashion retailer, runs a conversational Assistant that helps shoppers find clothes. After moving it from GPT-3.5 to GPT-4o mini, OpenAI reports a 23% rise in product clicks and a rise of more than 40% in wishlist additions versus the previous version.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
23%
Increase in product clicks versus the previous version of the Assistant1
5%
Fall in recommendations that users rated unhelpful after the move to GPT-4o mini1
12
x
Growth in Assistant traffic after moving from GPT-3.5 to GPT-4o mini, reached without spending significantly more1
What it took
The problem
Zalando sells a very large range of fashion. Shoppers often know the occasion, such as a wedding or a season, but not the product name. Zalando wanted people to describe what they need in their own words and get useful suggestions. As the company grew the idea to many countries, the first version showed limits. It ran on GPT-3.5, which followed detailed instructions poorly and gave generic answers to event and seasonal requests.
What they built
The Zalando Assistant is a chat feature in the app and website. A customer asks a question in everyday language and gets a written reply plus matching products, and can keep refining the request. Zalando announced a first beta in April 2023 for selected customers in Germany, Ireland, the United Kingdom and Austria, in English and German. OpenAI says the first version launched in four German and English-speaking markets.
The upgraded version runs on GPT-4o mini, a smaller and cheaper OpenAI model. It works in more than 20 languages, and OpenAI says it is available in all 25 Zalando markets.
How they did it
Zalando and OpenAI first improved how the Assistant is tested. The team set up separate checks for individual parts of the system, such as the step that routes a request and the step that writes the answer. It also improved the worked examples used to teach a reviewing model what a good or poor answer looks like.
With that in place, the team moved the Assistant from GPT-3.5 to GPT-4o mini. In two weeks it shifted half of the traffic to the new model, and it finished the rest soon after. OpenAI says the change opened up new languages such as French and Spanish and lowered delay and running cost.
Results
OpenAI reports that, compared with the previous version, product clicks in the recommendation carousel rose 23% and wishlist additions rose by more than 40%. Recommendations rated unhelpful by users fell by 5%. OpenAI also says Zalando grew Assistant traffic 12 times by adding markets without spending significantly more.
All of these figures are reported by the vendor and come without absolute numbers. Zalando says it plans more conversational features, such as answering detailed questions about gifts and family events.
All reported figures
| Measure | Figure | Citation |
|---|---|---|
| Increase in product clicks versus the previous version of the Assistant | 23%1 | |
| Increase in products added to the wishlist versus the previous version | 40%+1 | |
| Fall in recommendations that users rated unhelpful after the move to GPT-4o mini | 5%1 | |
| Growth in Assistant traffic after moving from GPT-3.5 to GPT-4o mini, reached without spending significantly more | 12x1 | |
| Markets where the Assistant is available | 251 |
Lessons for your project
Fix measurement before you change the model. Zalando first built tests for each part of the system, such as routing and answer writing, so it could judge an upgrade with evidence.
Switch traffic in steps. Zalando shifted half of the Assistant's traffic to the new model in two weeks and finished the move shortly after.
Choose the model for the job. Zalando needed 20 or more languages and reliable instruction following, so it replaced a model that gave generic answers to event and seasonal requests.
Track behaviour, not only satisfaction. Clicks and wishlist additions show whether recommendations lead shoppers to act.
Caveats
- The figures come from OpenAI's customer story, so they are vendor-reported. Zalando's own 2023 release does not contain them.
- The OpenAI page carries no date. The text says a 40%+ rise in wishlist additions in one place and 41% in another. We use the rounder headline figure.
- The comparison is with the previous version of the Assistant. The sources do not give absolute numbers, test design or time period.
- The deployed date is the month of Zalando's April 2023 announcement of the first beta, which was planned for a limited group of customers in the spring. The sources do not give the exact go-live date for the first version or for the 25-market rollout.
- The story says the move to GPT-4o mini cut latency and operating costs, but gives no cost figure.
Sources
- Zalando boosts the customer experience with its Assistant, powered by GPT-4o mini
OpenAI · Undated · Vendor case study · openai.com
- Zalando to launch a fashion assistant powered by ChatGPT
Zalando · 2023-04-19 · Press release · corporate.zalando.com
Summarized in our own words from the sources above. Spotted an error? See corrections.