Terminal-Bench-Science top score jumps 30% to 74.6% in four months, evals saturating
geoffwolfe · x · 2026-11-10
ReasonCore AI analyzes Claude Fable 5.2's 74.6% on Terminal-Bench-Science (top score was just 30% at the benchmark's August launch), highlighting three points:
- Resolution: only 70 tasks, task-clustered standard error of 4.3 points — Fable 5.2's 0.7-point gap to Mythos 5.2 is noise; its 6.5-point gap to GPT-6 Astra is about 1.5 SE.
- Supply: 920 proposals yielded just 70 merged tasks (7.6%) — hard, verifiable science tasks are being used up faster than they're built.
- Signal: near 75%, more tasks get solved on nearly every trial; the benchmark stops separating models and giving RL a gradient.
More from Models
- SemiAnalysis Launches GDPx Leaderboard: Claude Fable 5.2 Debuts on Top at 16.4% — scaling01 · 2026-11-10
- Anthropic caps GPU kernel optimization in Fable 5.2 training as a safeguard — community asks: pre or post nerf? — WolframRvnwlf · 2026-11-10
- Fable IS Mythos with more guardrails, users say as Anthropic calls them two models — repligate · 2026-11-10
- Researchers Test New Claude Models as Long-Horizon Research Assistants — chaitjo · 2026-11-10
- Open-source Mirage model trends on Hugging Face with zero-shot detection and VQA — Cactus-Compute · 2026-11-10
- 400M model handles game NPC judgment calls in ~0.1s on a 3060 Ti, Apache-2.0 — NoahPersaud · 2026-11-10