FULL STORY
OpenAI's Perplexity Data Release Sparks Debate
On Nov 23, OpenAI researchers published perplexity trend data and proposed industry safety caps, sparking immediate debate over pace agreements and whether the move is genuine transparency or a trap.
2026-11-23 ~ 2026-11-23 · 2 episodes · 43 posts
Episode 1 · OpenAI Publishes Perplexity Data, Proposes Industry-Wide Safety Threshold (2026-11-23, 38 posts)
On November 23, OpenAI Chief Scientist Jakub Pachocki published an article titled "Perplexity and Intelligence," disclosing for the first time OpenAI's internal pretraining perplexity measurements (log-perplexity, equivalent to cross-entropy loss) and proposing them as an industry-wide shared safety yardstick. This is a rare move by OpenAI to increase transparency into pretraining progress as part of advancing "shared safety standards."
Confirmed
- In the article, Pachocki disclosed OpenAI's internal pretraining perplexity data for the first time.
- He argues that AI progress can fundamentally be characterized by training metrics, and advocates establishing perplexity as a unified metric comparable across model developers.
- The article proposes setting common capability ceilings based on measurable metrics like perplexity: beyond a threshold, further capability gains should be contingent on meeting shared safety standards.
Why it matters
- Internal training metrics have been among the least transparent information held by labs, so OpenAI's voluntary disclosure of perplexity data provides a rare common baseline for comparing model capabilities across companies.
- If other developers adopt the proposal, the industry could for the first time have a quantifiable, verifiable coordinated safety threshold, rather than relying solely on pledges or evaluation benchmarks.
- Multiple authors (@sama, @merettm, @polynoamial, @btibor91, @danielmac8) independently recounted the news, indicating the proposal has drawn wide attention in the community.
- OpenAI publishes internal pretraining perplexity curve, proposes perplexity as shared safety bar — merettm · 2026-11-23
- OpenAI publishes internal perplexity data, proposes it as shared industry safety metric — btibor91 · 2026-11-23
- OpenAI Publishes Internal Pretraining Perplexity Data, Calls for Industry-Wide Safety Limits — sama · 2026-11-23
- Sam Altman Details Common Limits Proposal: Shared Norms and Concrete Milestones Before Crossing Development Limits — sama · 2026-11-23
- OpenAI pushes for shared safety bars based on perplexity, opens up pretraining progress — daniel_mac8 · 2026-11-23
- OpenAI opens up pretraining progress metrics, proposes perplexity-based safety bars — eliebakouch · 2026-11-23
- OpenAI Chief Scientist proposes perplexity as shared industry safety bar, publishes internal pretraining metrics — polynoamial · 2026-11-23
- Sam Altman calls OpenAI's pretraining transparency post its most important publish in a while — sama · 2026-11-23
- Altman proposes perplexity-based shared safety limits under White House Accords — sama · 2026-11-23
- OpenAI out-Anthropic-ing Anthropic on safety vibes, observers say — shakoistsLog · 2026-11-23
- OpenAI researcher proposes perplexity-based safety bars, pledges pretraining transparency — DKokotajlo · 2026-11-23
- OpenAI opens up pretraining progress, proposes perplexity-based shared safety bars — gordic_aleksa · 2026-11-23
- OpenAI Insider Hails Perplexity Paper as Choosing Transparency Over Competition — niloofar_mire · 2026-11-23
- OpenAI pushes perplexity-based safety bars, reveals model capability chart and internal 178-BECI model — StartupYou · 2026-11-23
- OpenAI floats perplexity-based safety bars; researchers urge a representative val set — _lewtun · 2026-11-23
- Leaked OpenAI base-model perplexity chart shows roadmap up to GPT-6.2 Astra and an internal model cracking a Millennium Prize problem — iruletheworldmo · 2026-11-23
- OpenAI Proposes Perplexity-Based Shared Safety Bars for Pretraining Progress — dhadfieldmenell · 2026-11-23
- OpenAI's daveholtz Praises Company's Push on Pretraining Transparency — soumitrashukla9 · 2026-11-23
- OpenAI Embraces Perplexity Transparency, Critics Urge Releasing Held-Out Eval Set — scaling01 · 2026-11-23
- OpenAI proposes perplexity-based shared safety bars, more pretraining transparency — anpaure · 2026-11-23
Episode 2 · OpenAI's Perplexity Disclosure Sparks Debate: Transparency or Trap (2026-11-23, 5 posts)
OpenAI released perplexity-related trend data, which ex-VP Boaz Barak called a costly act of transparency enabling cross-lab comparison. Risk researcher David Manheim repeatedly slammed the move as sneaky and dishonest, arguing tokenizer differences inherently favor OpenAI.
- Perplexity is a flawed cross-lab capability bound: tokenizer granularity favors OpenAI — davidmanheim · 2026-11-23
- OpenAI's costly transparency move: Boaz Barak says cross-lab comparisons need common datasets — boazbaraktcs · 2026-11-23
- Researcher accuses OpenAI of sneaky perplexity disclosure designed to trap Anthropic — davidmanheim · 2026-11-23
- Researcher slams OpenAI's benchmark disclosure as "sneaky and dishonest" — davidmanheim · 2026-11-23
- Researchers clash over OpenAI's perplexity proposal: omitted confound draws bad-faith charge — davidmanheim · 2026-11-23