Sharpening Tax paper: RL post-training trades pass@K coverage for pass@1 accuracy
kastnerkyle · x · 2026-11-03
The paper "Sharpening Tax in Post-Training" systematizes evidence for the hypothesis that RL post-training merely sharpens a base model's existing behaviors: pre-trained LLMs with a light inference harness have far lower pass@1 but often beat their post-trained counterparts in pass@K solution coverage given enough test-time budget.
Key findings:
- Post-training pushes tasks toward two extremes—always solved or never solved—improving sampling efficiency and consistency at the cost of coverage
- Proposes "Sharpening Tax," a diagnostic metric quantifying the loss in test-time scalability after post-training
- Evaluated across 14 base/post-trained model pairs from 4 families and 3 agentic benchmarks (42 cases); the tax is prevalent and estimable from a few rollouts
- Introduces posterior-tempered group sampling (PTGS), a plug-and-play Bayesian sampler that adapts temperature per prompt to estimated difficulty
Paper, code and models are publicly available.
More from Models
- ML4 Goes Open-Weight: Frontier Code Agent and Cyber Capabilities, Far More Compact Than Preview — _AndrewZhao · 2026-11-03
- OpenRouter: heavy users now median 6 model families, 2x in a year — sarahookr · 2026-11-03
- Killed by Google eulogizes AI Overview — 'not yet,' replies a user — gaganghotra_ · 2026-11-03
- DeepSeek's 'super agent' sparks debate: is the meat proxy era over? — gdechichi · 2026-11-03
- Only 3% of humans beat Claude Opus 5.5 at predicting what people click — teortaxesTex · 2026-11-03
- Nvidia ships Dory1-4B, a looped LLM that reuses 20 layers to rival much larger models — scaling01 · 2026-11-03