Labs sharing base-model perplexity metrics could mirror post-training evals

eliebakouch · x · 2026-11-23

eliebakouch praises markchen90's move to share base-model perplexity metrics as a "huge move": such metrics are only sensitive IP while exclusive; once everyone shares them they'll become like post-training evals but for base models. He adds more information can likely be retro-engineered from raw perplexity than from post-training evals.

Original post →

More from Models

Models channel →