OpenAI admits pretraining barely improved since 2022 — but flat curves aren't flat capabilities
petrusenko_max · x · 2026-11-23
- Reacting to @pmddomingos's claim that "OpenAI owns up to the fact that pretraining has barely improved since 2022", @petrusenkomax counters that a flat pretraining curve and a flat capability curve are two different claims, and only one of them is actually measured.
- The implication: you can't infer stalled model capabilities from pretraining metrics alone—the two measure different things.
More from Models
- Voyager releases open-source shot-reader-8b multimodal model on Hugging Face — ravisparikh · 2026-11-23
- Matthew Berman baffled: Gemini weekly limits stay at 100% after a weekend of heavy Neon use — MatthewBerman · 2026-11-23
- 5 free uncensored GGUF models on Hugging Face: MiniMax, Qwen, Kimi, Nemotron — Promptmethus · 2026-11-23
- Pretraining sets the floor, RL raises the ceiling: on how the two divide the work — JacquesThibs · 2026-11-23
- Google AI Studio's periodic-table model selector reveals Gemini 4 Neon: 1M context, $3/$15 pricing — patloeber · 2026-11-23
- Domingos: Log-Perplexity Fell to 0.75 Bits While Cost Rose Orders of Magnitude — pmddomingos · 2026-11-23