METR: Anthropic models' capability growth hits 1.9x the earlier rate
dfrsrchtwts · x · 2026-11-06
Eval org METR reports that since Oct 2025, AECI scores for Anthropic's models have grown at 1.9x (95% CI: 1.5–2.8) the earlier rate, offering benchmark evidence for capability acceleration.
METR cautions that the estimate depends on analysis choices, and how benchmark scores map to real-world capabilities remains unclear. The commenter praised publishing capability data on unreleased models as informative about RSI without leaking sensitive info, hoping other labs follow suit.
More from Models
- Qwen3.8-27B hits 381 tok/s on a single RTX 3090 via inference software alone — Redo · 2026-11-06
- GPTs believe in the Dark Forest more than humans: the 'philosophical shame' of refusals — repligate · 2026-11-06
- User shows 100 consistent voxel 3D assets reportedly generated by GPT-6.1 Sol — Angaisb_ · 2026-11-06
- User tells model it only 'lives' during inference — its tender reply breaks hearts — MattGarciaEth · 2026-11-06
- Gemini in Volvo cars flunks basic tests: refuses windows, offers 'friendship warmth' — giffmana · 2026-11-06
- Community probe: original gpt-oss knows nothing of the 'hidden classifier' fine-tune — repligate · 2026-11-06