Surge AI benchmark: smallest Claude Haiku beats the biggest GPT-6 models
echen · x · 2026-11-17
Blogger echen argues via a Jeff Dean analogy that a great engineer isn't one who writes correct passing code, but one who tells you when you're being stupid, weighs architectural tradeoffs, thinks ten steps ahead, and builds what you actually need. By that measure, "even the smallest Claude is a better Jeff Dean than the biggest GPT."
The attached Surge AI SUDO L7 chart backs the quip: Claude Sonnet 5.5 scores 62.2%, Opus 5.5 57.9%, and even Haiku 5.5 49.3% — all above GPT-6 Astra (47.7%) and GPT-6.1 Sol (47.4%).
Related event: Smallest Claude Beats Largest GPT-6 on Surge AI's New sudo L7 Benchmark(2 posts)→
More from Models
- Quantized Opus 5.5 says 'open' instead of 'run' — quantization quirks again — ryunuck · 2026-11-17
- OpenAI urged to release rumored model allegedly cracking Navier–Stokes millennium problem — haider1 · 2026-11-17
- Verus Proof Index: new formal-verification benchmark evaluates 25 models' agentic proofs — fhuszar · 2026-11-17
- Asking ChatGPT and Claude to play NeurIPS reviewers — they reject the papers — burny_tech · 2026-11-17
- Benchmark cheating debate: scaling a 3B open model to ~60B explains the scores — RexDouglass · 2026-11-17
- Physician slams OpenEvidence's "lowering all-cause mortality" claim; Eric Topol amplifies — EricTopol · 2026-11-17