Zomato
Zomato moves its Zia support chatbot to Llama 3 and reports 2x customer satisfaction
Zomato, the Indian food-delivery company, replaced menu-driven customer support with Zia, a conversational AI chatbot, and moved it from closed models to Meta's open-weight Llama 3 served by Together AI. Together AI reports a 2x improvement in customer satisfaction and 75% lower response times, now under 10 seconds.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
dropping by 75% to under 10 seconds
Average response time after the move, per Together AI1
What it took
Cost, team size and timeline were not disclosed in the sources.
The problem
Zomato delivers over a million food orders a day in India. Its old support flow was a menu of fixed options and scripted replies. It struggled with unusual situations, and simple questions such as "where is my order" often went to a human agent unnecessarily.
What they built
Zia is a conversational support bot. Customers type their problem in plain language. The bot can also act: it can add delivery instructions or cancel an order. Zomato tracked chat ratings (CSAT), the share of chats resolved without a human, cost, and response time, with a target of under 10 seconds.
The first version ran on closed models, including GPT-4, which raised cost and speed concerns. After Meta released Llama 3, an open-weight model whose files are published so companies can run or adapt them, Zomato moved to Llama 3 on Together AI's platform.
How they did it
The bot works in steps. It first classifies what data it needs, calls only the relevant internal systems, turns the data into plain language, and combines it with chat history and business guidelines in the final prompt. In parallel, a separate step spots possible actions and checks them against order status and customer history. The customer sees a confirmation pop-up before anything runs.
A policy layer sits on top. The model must tag any escalation to a person with a predefined reason, and that reason is checked against order data. Zomato added it after model-only decisions made containment swing between 80% and 50%.
Larger 70B models handle intent detection and critical tasks. Smaller 8B models write the chat replies. Shared serverless capacity covers normal traffic and dedicated capacity covers meal-time spikes of 10 to 50 times.
Every system change is reviewed by people. A share of live replies is sampled for human grading, and an AI judge is used too, on four measures: factual consistency, relevance, guideline compliance and whether the customer's pain point was resolved.
Results
Together AI reports a 2x improvement in CSAT scores and average response times dropping by 75% to under 10 seconds. It also says the system handled over 1,000 messages per minute at peak and more than 80,000 customers on one Mother's Day, and that more requests are now fully resolved by the AI at lower cost. It gives no figure for the cost saving.
What to copy
Put hard rules around the model for decisions with real consequences, such as escalation and refunds, and ask the customer to confirm before it acts. Test open models against your current ones on your own chats before switching.
Lessons for your project
Do not let the model decide alone when to hand a chat to a person. Zomato added a rule layer after model-only escalation swung containment between 80% and 50% from day to day.
Ask the customer to confirm before the bot acts. Zia shows a verification pop-up before an action such as a cancellation runs, and offers it only when order data supports it.
Size the model to the task. Zomato uses larger 70B models for intent detection and critical steps, and smaller 8B models for writing the chat reply.
Plan for lunch and dinner peaks. Zomato mixes always-on shared capacity with dedicated capacity for spikes of 10 to 50 times normal traffic.
Review changes with people and sample live replies for human grading, alongside an AI judge, and track four quality measures including factual consistency and guideline compliance.
Caveats
- The figures come from a Together AI customer story, which sells the infrastructure. Zomato's conference talk listing confirms the Zia project and its topics, but gives no figures. No independent source confirms them.
- The Together AI page prints no publication or go-live date. The 2024 year comes from its mention of the Fifth Elephant Conference 2024 talk, so it is the talk year, not a confirmed go-live date. The page does not say which Llama 3 release was used.
- The page does not say how CSAT was measured or what period the 2x and 75% changes cover. It also gives no cost figure, only that more requests are handled at lower cost.
Sources
- How Zomato built an AI customer support bot that doubled customer satisfaction and scaled to over 1,000 messages per minute
Together AI · Undated · Vendor case study · together.ai
- Scaling Customer Delight at Zomato using AI
The Fifth Elephant 2024 (HasGeek) · Undated · Conference talk · hasgeek.com
Summarized in our own words from the sources above. Spotted an error? See corrections.