Perplexity is a flawed cross-lab capability bound: tokenizer granularity favors OpenAI

davidmanheim · x · 2026-11-23

David Manheim argues that perplexity is a flawed metric to standardize on for cross-lab capability bounds: Anthropic's more fine-grained tokenizer yields inherently lower per-token perplexity, so any shared perplexity-based limit would let OpenAI push much further. A subtle but consequential measurement bias in lab safety agreement design.

Original post →

More from Models

Models channel →