Labs sharing base-model perplexity metrics could mirror post-training evals
eliebakouch · x · 2026-11-23
eliebakouch praises markchen90's move to share base-model perplexity metrics as a "huge move": such metrics are only sensitive IP while exclusive; once everyone shares them they'll become like post-training evals but for base models. He adds more information can likely be retro-engineered from raw perplexity than from post-training evals.
More from Models
- MemTensor launches MemT-1.0-Pro, a 554B memory-native model built with NVIDIA — _akhaliq · 2026-11-23
- Kimi K3.1 Pro on Cerebras hits 1375 TPS after 594 agent-run experiments — hyperparticle · 2026-11-23
- Perplexity is a floor metric, not a magic bullet, argues researcher — eliebakouch · 2026-11-23
- Aurora2-400M-A20M-QAT: a 0.5-bit MoE checkpoint shrunk from 799 MB to 27.5 MB — Tall_Abrocoma_3533 · 2026-11-23
- Leaked projection: OpenAI internal model at 0.732 perplexity, automated AI research by May 2027 — rickasaurus · 2026-11-23
- Aurora2-400M-A20M-SFT: chat fine-tune of a sparse MoE with only 19M active parameters — Tall_Abrocoma_3533 · 2026-11-23