Leaked Anthropic IPO roadshow slide shows superexponential pretraining compute efficiency gains
gleech · x · 2026-11-14
- A slide from Anthropic's IPO roadshow surfaced showing pretraining compute efficiency over time on a log-scale y-axis, revealing sustained superexponential algorithmic progress — the poster was surprised the company made it public.
- Key reading: pretraining was genuinely bad in the early LLM era; data curation and task design have improved dramatically since. "Mythos" wasn't a one-off deviation but a point on a superexponential curve — it could be retrained today with a fraction of the original compute.
- The author cautions this doesn't strictly imply a software-only intelligence explosion, since Anthropic has also raised R&D spending — but algorithmic improvements don't require creative genius and are on the verge of being automated.
A retweeting ML practitioner adds: early-era LLM pretraining really was that bad, and data/task curation has improved across the board.
More from AGI Musings
- SfN2026 minisymposium on embodied computation spans zebrafish whole-brain dynamics to NeuroAgents — tweetsatpreet · 2026-11-15
- Patient data that never leaves the hospital is the easiest AI project to get approved — MaziyarPanahi · 2026-11-15
- Blogger argues anti-AI sentiment is a psyop by old creative-industry gatekeepers — nptacek · 2026-11-15
- What happens when AI trains on AI content for 100 generations? — Huge-Conference-1193 · 2026-11-15
- LMU Dissertation Recasts Neural Networks as Cybernetic Error-Controlled Regulators on the Path to AGI — sebkrier · 2026-11-15
- The Strangest AGI Outcome: Humans Stop Being the Audience for New Knowledge — VraserX · 2026-11-15