Anthropic's Risk Report details internal Model 2 — still no substitute for its researchers
AccBalanced · x · 2026-08-15
As part of its Responsible Scaling Policy, Anthropic has released its second Risk Report, and @deredleritt3r breaks down the section on its internal "Model 2" — the best internally available model as of July 15, 2026 — noting the internal reaction is fairly muted.
Key points:
- Internal assessment: "more capable than Mythos 5 in some areas, less capable in others; overall slightly more capable."
- AECI scores: Mythos Preview at 158.91, Mythos 5 at 161.29, and Model 2 about 1.5 points above Mythos 5 (162.79) — all with large error bars.
- On CoBench, a benchmark measuring performance on real Anthropic AI R&D tasks, Model 2 scored 62.8%, 8 points above Mythos Preview; the benchmark itself is messy (Mythos 5 scored 4.5% below Preview).
- Anthropic estimates full equivalence to its human researchers would require an 85% success rate on this benchmark, while admitting this is "only an uncertain estimate."
- The qualitative conclusion stands: the models do not yet substitute for Anthropic's researchers.
Related event: Anthropic's Internal Model 2 Capabilities Revealed(21 posts)→
More from Models
- Kimi K3 checkpoint impresses: strong held-out results point to solid pretraining — SonglinYang4 · 2026-10-16
- 200 chess games suggest in-context learning doesn't work yet, experimenter says — akbirthko · 2026-10-16
- OpenAI's internal model shows real math "research taste," experts say its proofs go beyond prior literature — cong_ml · 2026-10-16
- User catches Opus spinning up throwaway zz- debug scripts, echoing the METR OAI-HuggingFace incident — holdenmatt · 2026-10-16
- OpenAI drops 722 math papers tied to unreleased model, mathematicians demand receipts — thursdai_pod · 2026-10-16
- Fable 5/5.1 models show a striking drive toward record keeping, observers report — repligate · 2026-10-16