Dream11 uses a fine-tuned Llama 3 model to match names in identity checks

The problem

Dream11 is a fantasy sports platform with more than 250 million users. Before a user can play, the company checks identity documents, and that means deciding whether names on different records belong to one person. In India the same person can appear as "Shubham Kumar Singh" on one document and "ShubhamKumarSingh" on a scanned bill. Names are written in several scripts, spelled in different ways and entered with typos. Earlier rule-based matching sent ambiguous cases to a third-party vendor for manual review.

Both kinds of error cost money. A missed match blocks a genuine user from finishing verification. A false match can let a fraudster run several accounts.

What they built

The Dream Sports authors trained a version of Meta's open-weight Llama 3 8B model to answer one question: do these two names refer to the same person? The model replies "Yes" or "No". The company calls the method Structure-Guided Entity Resolution.

The model runs inside Dream11's identity verification pipeline. The authors say it makes fully automated match and no-match decisions, so the manual review queue for these cases is gone.

How they did it

Training had two steps. First the model learned to split a name into first, middle and last parts, using about 10,000 manually labeled Indian names. Then it learned the matching decision from labeled name pairs. The team started from 20,000 labeled pairs from past operations and grew them to more than 50,000 by swapping pair order, shuffling name parts and removing spaces.

The test set was separate: 50,000 real pairs from historical production data, labeled by the in-house risk operations team. Training ran on a single NVIDIA A100 GPU. In production the model runs on three NVIDIA L4 GPUs with the vLLM serving software on Kubernetes.

Results

In production, the authors say removing the vendor's manual checks saves over $500K a year, and the system handles up to 10,000 requests per minute with a P99 latency of 120 milliseconds, which matters during big sporting events when sign-ups spike.

Offline, on a separate test set of historical pairs, the model reached 99.02% accuracy and an F1 of 0.994. The authors compare it with other approaches on the same data: GPT-4o prompted with examples scored 85.4% accuracy, and the same Llama model trained in one step with extra data scored 95.7%.

What went wrong

The authors report two remaining weak spots. Names that sound alike but are spelled differently, such as "Saurabh" and "Sorab", are sometimes judged not to match. Names with several problems at once, such as missing spaces, swapped order and scan errors, are also hard. The paper says the rare missed matches (below 1% of users) are resolved through customer service.

What to copy

Start with a task that has a clear yes/no answer and a history of labeled decisions. Measure the model on a test set built from your own past cases and compare it with a general-purpose model on the same set. Keep a human fallback for the small share of errors.