OpenAI Embraces Perplexity Transparency, Critics Urge Releasing Held-Out Eval Set
scaling01 · x · 2026-11-23
OpenAI researcher merettm announced greater transparency into pretraining progress in "Perplexity and Intelligence", arguing for shared safety bars based on perplexity and gating further progress on understanding AI behavior. Reposter oneillc urged OpenAI to go further and release its held-out perplexity evaluation set, claiming it would instantly leapfrog Anthropic in safety contributions. Jakub's quoted view: cross-lab comparison is hard due to differing eval sets and data mixtures, but transferable comparison methods are possible if transparency becomes a shared goal.
More from Models
- Influencer's overall model ranking puts opus 5.5 on top, weighing cost and speed — MatthewBerman · 2026-11-23
- Gemini 4 Neon Builds a First-Person Murder Mystery Game with 16 Walk-In Bars — venturetwins · 2026-11-23
- Leaked OpenAI base-model perplexity chart shows roadmap up to GPT-6.2 Astra and an internal model cracking a Millennium Prize problem — iruletheworldmo · 2026-11-23
- Fine-tuned Inkling 2 Mini routes tickets with early exit: median latency 136.8ms to 81.4ms at 0.07% accuracy cost — eltokh7 · 2026-11-23
- OpenAI's Michelle Pokrass confirms GPT-4.1 wasn't a fresh pretrain, just a new mid-train — JasonBotterill · 2026-11-23
- 55-Page Riemann Hypothesis Proof Guide, De-Formalized from Lean with Gemini and GPT — chaumian · 2026-11-23