Amazon swaps Sonnet 4.6 for a decision model in ticket labeling, cutting cost by 98%
victormustar · x · 2026-11-10
Two Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider, a decision model released a month earlier, to label tickets in one of the world's largest internal ticketing systems.
Key points:
- The old setup ran an LLM agent with a long system prompt on every ticket, generating JSON and reasoning over 20 fixed categories (escalations, capacity requests, alarms, security findings, etc.).
- A decision model can't generate text: it takes a state (the ticket) plus a question with candidate options (the label catalog) and returns a probability and confidence score per option — a natural fit for labeling.
- They started with Strands Decider 2B, but the E4B Gemma 4 build performed clearly better on a held-out test set and was adopted instead, followed by tuning the confidence threshold for applying labels.
- The swap matched Sonnet 4.6's quality at 98% lower cost, saving millions of dollars a year in token spend.
The takeaway echoes a broader engineering pattern: for classification tasks, use an LLM to generate labeled data, train a small specialized decision model, then replace the LLM.
More from coding & agent
- Skills vs linked docs: where should AI agents' knowledge live? — IllustriousFault6218 · 2026-11-10
- Fable 5.2 models a Bugatti Tourbillon in Blender from a single prompt in 2.5 hours — socialwithaayan · 2026-11-10
- Pake: Rust-based tool packs any webpage into a lightweight desktop app, 539K downloads — Aiden_Tech_Ai · 2026-11-10
- Frontier3DBench: 145 tasks test whether coding agents can replace 3D vision pipelines — Aiden_Tech_Ai · 2026-11-10
- LASIK open-sourced: strips anti-AI obfuscation, cutting 21M tokens to 17k — morphh7 · 2026-11-10
- Human-in-the-loop is easy; keeping the human attentive is hard — 4 metrics to test if your AI approvals mean anything — alex_verem · 2026-11-10