Infrathon Benchmark Tests LLM Agents on Real-World AI Infra Optimization; Claude Opus 5.5 Leads
RulinShao · x · 2026-11-17
Researchers from UW, Allen AI, Stanford and others launched Infrathon, a benchmark of 25 real-world, open-ended AI infrastructure optimization tasks with 12–24-hour budgets (150 agent runs, 10,000+ GPU-hours). Agents edit systems, pass a Controller code review, and are scored by a Judge on held-out workloads. Claude Opus 5.5 leads all six models with a mean rank score of 91.2, winning 17/25 tasks — including boosting mean KV-cache compression from 6.82× to 19.86× while improving video quality.
More from Infra
- Power Is a Red Herring for AI: Frontier Compute Efficiency Grows 10x+ Per Year — teortaxesTex · 2026-11-17
- Topos Bio teams with NVIDIA cuPotential to boost molecular dynamics throughput up to 3.6x — AllThingsApx · 2026-11-17
- PyTorch Day Japan 2026 set for Dec 10 in Tokyo: edge deployment and kernels — PyTorch · 2026-11-17
- New BIS rule forces restricted-region chip designers to win BIS approval, ending foundry transistor-count verification — terryyuezhuo · 2026-11-17
- Baseten hits ~2,500 tok/s on DeepSeek v4.1 Flash with Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-17
- Microsoft outage hits airlines and banks, sparking global cyber incident fears — mallow610 · 2026-11-17