OpenAI chief scientist: model at log-perplexity ~0.76 cracks open math problems, 2027 acceleration plausible
Singularitarian · x · 2026-11-23
In "Perplexity and Intelligence", OpenAI chief scientist Jakub Pachocki says OpenAI has spent over a year evaluating models with open math problems. Once upgraded to a model of log-perplexity 0.76, it started producing groundbreaking results — long-standing math questions resolved one after another in a single Slack thread, which he found deeply surreal.
He extrapolates that a similarly sudden transformation is plausible in deep learning research, possibly yielding an avalanche of fundamental pretraining advancements as soon as 2027. He expects AI to help alignment and monitoring too, but doubts progress will be fast enough to unlock pretraining's full potential. OpenAI is also committing to more pretraining transparency, arguing for shared safety bars based on perplexity and gating further progress on understanding AI behavior.
More from AGI Musings
- Writer Predicts LLMs Will Train on Bot Output as Human Writing Dwindles — TuhinChakr · 2026-11-23
- Apple's appenz: agent swarms are filling in NVIDIA's CUDA moat — appenz · 2026-11-23
- What comes after know-how? LeCun weighs in on AI redefining intelligence — alexbilz · 2026-11-23
- METR-fit extrapolation implies 66-hour task horizons for 6.2 Astra, 128 hours for internal model — eliebakouch · 2026-11-23
- Uber burned its annual AI budget in 4 months with no visible improvement — yaroslavvb · 2026-11-23
- Post-training may exceed 75% of FLOPs, undercutting perplexity-based safety bars — stochasticchasm · 2026-11-23