OpenAI Discloses 5 Years of Perplexity Data in Bid for Shared AI Safety Baseline
markchen90 · x · 2026-11-23
- OpenAI researchers argue that frontier labs each writing their own safety cases and guardrails makes race dynamics hard to escape, and call for common limits backed by shared safety cases.
- The missing piece is a future-proof yardstick for AI progress. They propose perplexity as one candidate and take a first step by publishing OpenAI's own pretraining perplexity results from the past five years.
- The authors admit this is uncomfortable: those numbers were long treated as sensitive IP, and third-party evaluators may need to measure perplexity over common datasets. The goal is to spark a norm of sharing key metrics that common safety cases can build on.
- Cited context: merettm's piece "Perplexity and Intelligence" argues for common limits based on perplexity, with further progress gated on our ability to understand AI behavior.
More from AGI Musings
- Writer Predicts LLMs Will Train on Bot Output as Human Writing Dwindles — TuhinChakr · 2026-11-23
- Apple's appenz: agent swarms are filling in NVIDIA's CUDA moat — appenz · 2026-11-23
- What comes after know-how? LeCun weighs in on AI redefining intelligence — alexbilz · 2026-11-23
- METR-fit extrapolation implies 66-hour task horizons for 6.2 Astra, 128 hours for internal model — eliebakouch · 2026-11-23
- Uber burned its annual AI budget in 4 months with no visible improvement — yaroslavvb · 2026-11-23
- Post-training may exceed 75% of FLOPs, undercutting perplexity-based safety bars — stochasticchasm · 2026-11-23