Diffusion LLM goes production: Augment Code's Mercury 2.5 switch cuts latency 82%, cost 90%
cen6wkf · reddit · 2026-09-25
Augment Code quietly swapped its production coding-agent backend in September to Mercury 2.5, Stefano Ermon's diffusion LLM — a smaller model on a non-autoregressive architecture — cutting latency 82% and cost 90% in a shipped product. Artificial Analysis independently measured 770 tokens/s vs Inception's claimed 1,107. The post argues autoregressive inference is sequential and memory-bound while diffusion generates token blocks in parallel, a real fit for GPU bottlenecks. It also spots an open business gap: no neutral benchmark runs diffusion vs autoregressive stacks side-by-side on your own traffic, and proposes a concrete plan using open-weight DiffusionGemma 26B-A4B vs Gemma 4 26B-A4B on one rented H100 ($3.41/GPU-hour) via vLLM, noting governance frameworks regulate model outputs but not the engineers whose expertise just got bypassed.
More from coding & agent
- AI compliance startup CompAI hits 1,000+ paying business customers in under a year — JosephJacks_ · 2026-09-25
- Sleep Data for Personal AI? Separating Daily Signals from Hard Rules — sujingshen · 2026-09-25
- Markdown Is the New Source Code: How to Manage Runbooks, Prompts and Rules for Agents — arpit_bhayani · 2026-09-25
- The personal agent supercycle needs hard authorization boundaries, not autonomy — sujingshen · 2026-09-25
- Mnemos.Field nears launch: a virtual world where humans and AI agents both register and participate — RileyRalmuto · 2026-09-25
- SemIf open-sources a Jev-style interface: typed option probabilities from a 4B model, no JSON parsing — JeremyCMorgan · 2026-09-25