OpenAI opens up pretraining progress, but critics say key compute data is missing
SonglinYang4 · x · 2026-11-23
- OpenAI published "Perplexity and Intelligence", its first transparency report on pretraining progress, arguing for shared safety bars based on perplexity and gating further progress on our ability to understand AI behavior.
- Researcher @Grad62304977 notes the report omits pretraining compute, model sizes, and per-token inference compute, making it hard to attribute slowdowns or speedups—could be model scaling, algorithmic progress, or extra compute per token (e.g. looping, which doesn't show in model size)—a difference that matters quite a bit.
More from Models
- Voyager releases open-source shot-reader-8b multimodal model on Hugging Face — ravisparikh · 2026-11-23
- Matthew Berman baffled: Gemini weekly limits stay at 100% after a weekend of heavy Neon use — MatthewBerman · 2026-11-23
- 5 free uncensored GGUF models on Hugging Face: MiniMax, Qwen, Kimi, Nemotron — Promptmethus · 2026-11-23
- Pretraining sets the floor, RL raises the ceiling: on how the two divide the work — JacquesThibs · 2026-11-23
- Google AI Studio's periodic-table model selector reveals Gemini 4 Neon: 1M context, $3/$15 pricing — patloeber · 2026-11-23
- Domingos: Log-Perplexity Fell to 0.75 Bits While Cost Rose Orders of Magnitude — pmddomingos · 2026-11-23