Naive extrapolation nails AI task horizons but blows up on cyber vulnerabilities
maksym_andr · x · 2026-10-21
The author argues that naive extrapolation works remarkably well for some AI capability metrics (scaling laws, METR task time horizons — GPT-2's 4 seconds climbing log-linearly to 16-hour tasks on Claude Opus 4.6 / Mythos Preview) while completely breaking down on others: Epoch AI data shows serious CVE disclosures exploding from 300/month to nearly 1,900 right after Claude Mythos Preview's release. Most disagreements about future AI progress, he argues, stem from conflating these two regimes — baseline capability trends extrapolate smoothly, but cyber exploitation and high-severity agentic incidents behave like phase transitions.
More from AGI Musings
- Toby Ord sanity-checks METR time horizons: OpenAI's number may run 14 months behind expectations — tobyordoxford · 2026-10-22
- LMArena Talks to Amazon AGI Scientist on Evals as the Real Differentiator — arena · 2026-10-22
- 'You Don't Count Your Threads': a nuanced take on agent autonomy and swarms — _AustinCalvert_ · 2026-10-22
- Lee Cronin: Chemistry is where computationalists learn the universe isn't a computer — shyamalanadkat · 2026-10-22
- Economist Paul Novosad: AI makes faking effort free — and that breaks recommendation letters — paulnovosad · 2026-10-22
- VraserX: humanity's future is merging with superintelligent AI, not competing against it — VraserX · 2026-10-22