T-Tech
T-Tech automates 45% of customer support sessions with a fine-tuned Qwen model
T-Tech trained a Qwen2.5 7B model on how human operators handle customer chats in a business process platform, then let it run low-risk steps on its own. In production the authors report it fully automated 45% of sessions and cut operator active time by 39% against human-only handling, with no statistically significant drop in resolution quality.
Key figures
Copied exactly from the sources. Copy any figure with its citation.
45%
Share of support sessions the system handled end to end without operator intervention, in production1
227.37
s to 139.15 s (-39%)
Average operator active time per customer, human-only workflow vs selective automation (A/B test, 15.5k customers per group, 7 days)1
124.67
s to 104.72 s (-16%)
Average operator active time per customer, existing rule-based chatbot with scripted steps vs the new system (A/B test, 77k customers per group, 32 days)1
187.66
s to 150.13 s (-25%)
Average operator active time per customer, automation with mandatory final confirmation by the operator vs automation without it (A/B test, 17k customers per group, 7 days)1
What it took
The problem
Customer support at scale runs on enterprise workflow software, where operators click through screens to solve each case. Automating this with hand-written scripts is slow to build and breaks whenever the screens or processes change. General-purpose AI agents make many model calls per case, so small errors add up and the cost is high.
What they built
T-Tech built an AI agent that works inside its business process management (BPM) platform, the software operators use to handle support chats. The agent reads a structured view of the screen and proposes the next action, such as clicking a control, transferring a chat or sending a message. A second model, called a critic, judges how confident the agent is. Only high-confidence steps run on their own. For anything uncertain the agent hands the case back to an operator and carries on from the updated screen.
The model is Alibaba's open-weight Qwen2.5-Instruct-7B, fine-tuned on T-Tech's own data. The authors chose the 7B size because it gave the best balance of speed and throughput for their serving budget.
How they did it
The rollout has three stages. First the platform only logs what operators do. Then the model runs as a copilot: operators accept its suggestion or correct it, and that feedback trains the confidence model. Finally the system runs high-confidence steps in the background. The authors say a new process reaches selective automation in about two weeks without hand-written rules.
Support traffic is split into issue types, each with its own models, so a drift in one area can be isolated. If quality signals slip for a slice, automation switches off there and the system returns to copilot mode to collect new feedback. Training data is anonymized before use.
Results
The authors report that the system fully automated 45% of sessions in production. In an A/B test with 15.5k customers per group over 7 days, average operator active time per customer fell from 227.37 seconds to 139.15 seconds (-39%) against human-only handling. Against the existing chatbot with scripted steps (77k customers per group, 32 days), it fell from 124.67 to 104.72 seconds (-16%). Blind operator review found no statistically significant difference in resolution quality in either test.
Operators accepted routine actions such as transferring a chat 91.7% of the time, but free-text replies only 50.81% of the time. The authors name natural-language messaging as the main limit on further automation.
What went wrong
Operators rejected a share of suggestions. The authors analyzed those rejections and found that 37.2% were acceptable actions that the operator simply chose to do differently, for example softening the wording of a message. Cases outside the training data, such as new screen elements or unusual requests, are a known weak point, which is why the system defers to people when unsure.
What to copy
Run the system as a copilot first and use the operators' accept and correct clicks as training data. Automate low-risk, repetitive steps before anything customer-facing. Track operator time per customer in a controlled test, and add a switch that returns a case type to human handling when quality drops.
All reported figures
| Measure | Figure | Citation |
|---|---|---|
| Share of support sessions the system handled end to end without operator intervention, in production | 45%1 | |
| Average operator active time per customer, human-only workflow vs selective automation (A/B test, 15.5k customers per group, 7 days) | 227.37 s to 139.15 s (-39%)1 | |
| Average operator active time per customer, existing rule-based chatbot with scripted steps vs the new system (A/B test, 77k customers per group, 32 days) | 124.67 s to 104.72 s (-16%)1 | |
| Average operator active time per customer, automation with mandatory final confirmation by the operator vs automation without it (A/B test, 17k customers per group, 7 days) | 187.66 s to 150.13 s (-25%)1 | |
| Share of the system's suggested free-text chat messages that operators accepted in copilot mode (averaged over one week) | 50.81%1 |
Lessons for your project
Start with a copilot before automating. Operators accepted or corrected each suggestion, and that feedback taught the system which steps were safe to run alone.
Let the system say when it is unsure. It runs only high-confidence steps and hands the rest to a person, and it switches itself off for a type of issue if quality signals slip.
Automate procedural clicks first, wording last. Operators accepted routine actions far more often than free-text replies, so the team kept sensitive actions under human approval.
Compare against more than one baseline. The team tested against human-only handling and against its existing chatbot, which gave a more honest picture of the gain.
Caveats
- All figures come from the authors' own paper at the ACL 2026 industry track. No outside party has confirmed them.
- The paper names T-Tech as the authors' employer and says the system serves production traffic. It does not name a bank brand or say which products the support chats cover; the only mention of a bank is in the prompt printed in the appendix, which refers to a bank's support chat.
- The paper does not state the company's country or region, so the region is shown as global, the schema's catch-all. The only language hint is the appendix baseline prompt, which tells the model to converse in Russian.
- The 45% automation share is the paper's production figure with no test period or sample size attached. The time reductions come from the three A/B tests listed, which measure operator active time, not total resolution time.
- Resolution quality was scored by blind operator review. The paper reports no statistically significant difference from the baselines, which is not proof of identical quality.
- The paper does not say when the system went live. The deployed date is the proceedings month (July 2026).
- The authors say automation is limited to low-risk actions. Sending messages and irreversible account changes need explicit human approval, so the gains do not reflect unconstrained autonomy.
- The authors state that the system is meant to scale support capacity, not to cut headcount.
Sources
- Learning Selective LLM Autonomy from Copilot Feedback in Enterprise Customer Support Workflows
ACL 2026 Industry Track (T-Tech authors) · 2026-07 · Research paper · aclanthology.org
Summarized in our own words from the sources above. Spotted an error? See corrections.