Convirza serves 60 call-scoring models from one Llama 3 base and cuts cost 10x versus OpenAI

The problem

Convirza helps enterprises analyse customer calls and score how well agents perform. It processes millions of calls a month and extracts information into scorecards. Its older system used Longformer models (a type of text-analysis model) trained on over 60 indicators. Training took 9 to 24+ hours per iteration. Each model also needed its own compute, which cost $500 to $1,500 per indicator per month and limited the ability to add custom indicators for customers.

What they built

Convirza trained a separate small add-on (a LoRA adapter) for each of its 60 indicators and serves them all from one Meta Llama-3-8b base model. Llama-3-8b is an open-weight model, meaning its files are published and can be adapted by a company. Fine-tuning means further training on a company's own labelled examples. The adapters run on Predibase's managed platform, built on LoRAX, an open-source tool that serves many adapters off a single base model.

How they did it

Convirza chose a managed platform rather than building multi-adapter serving itself, because the load is highly variable and the engineering work would have been large. In the first month it trained and deployed adapters for 20+ indicators. The platform scales GPUs automatically, so Convirza can scale up to double-digit A100 chips at peak and back down when calls slow.

Results

According to Predibase, Convirza saw a 10x cost reduction versus OpenAI, an 8% higher average F1 score than OpenAI (F1 balances missed cases against false alarms) and 80% higher throughput than OpenAI. Compared with its earlier Longformer models, throughput is 4x higher. Mean inference time is sub-second, and Convirza's VP of AI says average response time stays under two seconds across 60 adapters.

What to copy

If your AI feature needs many tailored variants, test an adapter-per-task design on one shared base model before paying for per-feature compute.