Meta Quantifies the 'Sharpening Tax': RL Post-Training Trades pass@K Coverage for Accuracy
meta · hf · 2026-10-02
Meta's study confirms and refines the 'sharpening' hypothesis for LLM RL post-training: on agentic tasks, pre-trained models with a light inference harness often beat their post-trained counterparts in pass@K solution coverage despite far lower pass@1. Post-training pushes tasks toward always-solved or never-solved extremes, trading coverage for sampling efficiency. The team proposes Sharpening Tax, a diagnostic metric validated across 14 base/post-trained pairs from four families and three agentic benchmarks, estimable from a few rollouts. They also introduce posterior-tempered group sampling (PTGS), a plug-and-play Bayesian sampler that adapts temperature per prompt to estimated difficulty, paying a smaller tax while improving single-shot accuracy in two agentic RL environments.
More from Research
- LEV: Looped decision model lifts 2.6B accuracy 63.2% to 76.2% with adaptive compute — jiqizhixin · 2026-11-19
- Aurora2-400M-A20M: a sparse MoE pretrained from scratch in 36 hours on one RTX Pro 6000 — Tall_Abrocoma_3533 · 2026-11-19
- Researcher calls METR/Anthropic '2x speedup' RSI analysis statistically dubious — jessi_cata · 2026-11-19
- Linear Probes Could Reveal Which Persona a Base Model Is Playing, Researchers Suggest — cephaloform · 2026-11-19
- ETH's Anymal robot completes 10km autonomous alpine hike with VLM-guided ROAM system — ChongZzZhang · 2026-11-19
- OpenVax codebase grows from 78K to 565K lines in 15 months — iskander · 2026-11-19