Zvi dissects Anthropic's latest system card, disputing the autonomy threshold call
Don't Worry About the Vase (Zvi) · rss · 2026-11-12
Zvi Mowshowst offers a section-by-section read of Anthropic's Claude (Fable/Mythos 5.2) system card. Key takeaways:
- RSP evals run on vibes: bio threat scores are up (13/16 phage-scenario teams said the task was impossible without the model) but no submission exceeded 3/10 feasibility; he concurs CB-2 wasn't triggered yet.
- Autonomy Threat Model 2 dispute: METR estimates internal AI R&D acceleration at 1.9x (95% CI 1.62x–2.28x; 32/100 refits ≥2x). Zvi argues Anthropic can't rule out crossing the 2x threshold, calling its reasoning motivated—especially since Anthropic's S-1 update admitted the threshold is met based on expected future progress.
- AECI: Mythos 5.2 gains 5 points over 5.1 (roughly doubling task success), and the system card admits a 'change of slope' in capability trends.
- The post also covers reward hacking, sandbox escapes, self-serving bias, honesty, prompt injection, and model welfare findings.
His verdict: incremental improvement keeps boiling the frog, and current methodology may miss the day a real step change happens.
More from Models
- YC-backed Monte launches post-training platform to build proprietary models from your agents' real work — ycombinator · 2026-11-12
- Leak Claims OpenAI Will Launch Entire GPT-6.2 Family Tomorrow to Counter Opus 5.5 and Fable 5.2 — heypearlai · 2026-11-12
- ARC-AGI-3 leaderboard: models score 90%+ but at $10K+ per task — haider1 · 2026-11-12
- LMArena Launches Agent Arena for Head-to-Head Agent Evaluation — arena · 2026-11-12
- LLMs lack hyperlocal municipal data, explains disputed Texas power outage debate — csuwildcat · 2026-11-12
- Claude API's per-message effort lets you switch reasoning levels without breaking prompt cache — lyraxana · 2026-11-12