OpenAI Chief Scientist proposes perplexity as shared industry safety bar, publishes internal pretraining metrics

polynoamial · x · 2026-11-23

OpenAI chief scientist Jakub Pachocki argues in "Perplexity and Intelligence" that perplexity (cross-entropy loss) should become a common metric for comparing model developers and setting shared safety limits, with progress beyond such limits gated on our ability to understand AI behavior.

As a first step, OpenAI is publishing its internal pretraining perplexity measurements. Pachocki acknowledges this gives competitors a clearer picture of OpenAI's standing but says the urgency of industry-wide shared safety measures outweighs competitive concerns. He frames AI progress as building better compressors on large corpora, then eliciting latent capabilities via prompting, finetuning, and RL.

Related event: OpenAI Publishes Pretraining Perplexity Data, Proposes Industry Safety Threshold(41 posts)→

Original post →

More from Safety

Safety channel →