Leaked OpenAI loss curve suggests GPT-6.2 Astra solved a Millennium Prize problem, internal model already lower
kimmonismus · x · 2026-11-23
kimmonismus parses OpenAI's pretraining loss curves: the "solved a Millennium Prize math problem" capability line aligns with GPT-6.2 Astra (0.75-0.76 perplexity), and an unreleased internal model is already 0.03 lower — roughly the GPT-5 → GPT-5.4 jump. Chief scientist Jakub Pachocki says smaller perplexity gains mean bigger intelligence leaps near the floor, expects new capabilities around 0.68-0.7, and hints pretraining research could see an avalanche like math's as soon as 2027. The Millennium result came from an early-September internal model; less than 3 months later a newer one exists. OpenAI says AI itself helped design its newest models, with steep acceleration expected in 2026.
The chart maps thresholds to capabilities: below 0.9 benefits from thinking, 0.86 useful coder, 0.82 cybersecurity expert, 0.75 Millennium-level math. Emergent capabilities in cybersecurity and math appeared once losses fell below 0.8 — not obvious a priori.
Related event: Leaked chart suggests big perplexity jump in OpenAI's internal models(3 posts)→
More from AGI Musings
- David Sacks: Anthropic pitching recursive self-improvement to IPO investors, and he's worried — DavidSacks · 2026-11-23
- Dylan Patel: efficient pretraining makes post-training misalignment exponentially easier — dylan522p · 2026-11-23
- Harari to Suleyman: 'If they are really gods, they will not remain your slaves' — PeterBowdenLive · 2026-11-23
- Two-person lab GSI claims 250+ breakthroughs across 8 fields with custom AI-built solvers, not brute-force compute — udmrzn · 2026-11-23
- iamtrask: Great writing is constrained by IP access, proofs only by compute — iamtrask · 2026-11-23
- Daniel Litt: the Lean-formalized Riemann proof is likely correct, no human understands it yet — littmath · 2026-11-23