Grab
Grab uses GPT-4 to document data tables and speed up data discovery
Grab's analysts struggled to find the right dataset among more than 200,000 tables, because few were documented. Grab used GPT-4 to draft table documentation and tuned its internal search tool, Hubble, and reports the share of key tables with documentation rose to about 90%.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
94%
, a 12 percentage point increase
Hubble search click-through rate at the end of the first half of 2024 (monthly average was 82% before)1
increased by 70 percentage points to ~90%
Documentation coverage of the most frequently queried (P80) data lake tables, after GPT-4 generated docs (20% before)1
~95%
Users who found the generated documentation useful, per user feedback1
73%
, a 17 percentage point increase
Survey respondents who found it easy to discover datasets, after the improvements1
What it took
Cost, team size and timeline were not disclosed in the sources.
The problem
Grab's data lake held more than 200,000 tables, and finding the right one was slow. Only 20% of the most frequently queried tables had documentation. Staff leaned on colleagues in Slack to find data, and a survey the year before found that 51% of data users took multiple days to find the dataset they needed. The internal search tool, Hubble, matched keywords only and could not use the context in a question. Its monthly average click-through rate was 82%, so in 18% of sessions people left without opening any dataset.
What they built
Grab ran three linked efforts in the first half of 2024. It tuned Hubble's keyword search, used GPT-4 to generate table documentation, and launched HubbleIQ, a chatbot that answers data-discovery questions in plain language inside Hubble and in Slack.
How they did it
For search, the team studied click data and interviewed users. It boosted the most-used tables and certified tables, pushed down deprecated ones, hid irrelevant datasets and simplified the screen.
For documentation, Grab built a generator that reads a table's schema (its column layout) and sample data and asks GPT-4 to write a description. The prompt was refined over several rounds of feedback from the data owners. A "generate" button let owners create documentation themselves. Grab also pre-filled documentation for the most critical tables and asked owners to review it. Until an owner accepted or edited the text, data users saw an "AI-generated" tag.
For the chatbot, Grab connected Hubble to Glean, an enterprise search tool it already used, and built HubbleIQ with Glean Apps, which Grab describes as a language model with a custom system prompt that can read the catalogued datasets. Likely conversational searches show HubbleIQ results first. Grab later added HubbleIQ to Slack and planned to place it in analytics teams' question channels.
Results
Search click-through rose to 94% over the half, a 12 percentage point increase. Documentation coverage for the most queried tables rose by 70 percentage points to about 90%, and about 95% of users who gave feedback found the generated documentation useful. A follow-up survey found 73% of respondents found it easy to discover datasets, up 17 percentage points. Grab also reports that Hubble reached an all-time high in monthly active users.
Lessons for your project
Fix the foundation before adding a chatbot. Grab first raised documentation coverage and search quality, then built the conversational bot on top of that content.
Let the AI draft and the data owners approve. Grab tagged generated documentation as AI-generated until a producer accepted or edited it.
Start with a measured baseline. Click-through rate, documentation coverage and a user survey gave Grab clear before-and-after numbers.
Buy the plumbing when speed matters. Grab built its chatbot on an enterprise search tool it already used instead of building retrieval from scratch.
Caveats
- All figures are Grab's own, from one engineering blog post. No outside party confirms them.
- The post gives no go-live date. Its goals were set for the first half of 2024, so only the year is shown as the deployed date; the publication date is 26 September 2024.
- The post does not name the model behind the HubbleIQ chatbot, which was built on Glean Apps. GPT-4 is named only for the documentation generator.
- The post gives no results for HubbleIQ itself, such as answer quality or time to find a dataset. Its stated aim of cutting discovery from days to seconds is a goal, not a result.
- The post gives no cost or team size, and does not say how many tables were documented or how many users were surveyed.
Sources
- Enabling conversational data discovery with LLMs at Grab
Grab Engineering Blog · 2024-09-26 · Company blog · engineering.grab.com
Summarized in our own words from the sources above. Spotted an error? See corrections.