Meta's "Sharpening Tax": RL Post-Training Trades pass@K Coverage for pass@1 Accuracy
Meta Superintelligence Labs researchers (Changdae Oh et al.) propose the "Sharpening Tax" in a new paper, empirically validating the sharpening hypothesis in LLM post-training and offering corrections, showing that RL post-training boosts single-shot accuracy while narrowing the model's solution space.
Confirmed
- Core finding: on agentic benchmarks including BFCL v4 multi-turn, ACEBench and WebShop, base models with only a lightweight reasoning harness lag far behind RL post-trained versions on pass@1, yet under a sufficiently large test budget their pass@K solve coverage often surpasses post-trained models (m1, m2, m4, m5).
- Measured data: after 128 attempts, the share of tasks the base model "occasionally solves" drops from 87.6% to 30% (m4).
- The researchers attribute this to the sharpening hypothesis: post-training amplifies behaviors already present in the base model, sharpening existing capabilities rather than teaching new agentic skills (m3, m4).
- The authors argue that a base model with capable structured-action and tool-interface-following abilities can serve as a capable agent, making agentic tasks the most effective testbed for the hypothesis (m2, m4).
- Meta's official account also published the work, framing it as trading pass@K coverage for single-shot accuracy with proposed corrections (m5).
Why it matters
- The work quantifies the test-time-scaling cost of post-training—pass@K coverage traded for pass@1 precision—offering a new evaluation lens for post-training strategies and inference-budget allocation.
- If the sharpening hypothesis holds, agent workflows relying on large sampling and search should weigh the capability-narrowing risk of RL post-training more carefully.
2026-10-02 ~ 2026-10-04 · 5 related posts
Primary sources
- Sharpening Tax: Does post-training teach LLMs new agentic skills or just sharpen old ones? — SharonYixuanLi · 2026-10-02
- [source] Meta Quantifies the 'Sharpening Tax': RL Post-Training Trades pass@K Coverage for Accuracy — meta · 2026-10-02
- [source] Meta Paper: Post-Training Boosts pass@1 but Shrinks LLM Agents' pass@K Solution Coverage — iScienceLuvr · 2026-10-02
- Sharpening Tax: Post-Training Cuts "Sometimes Solved" Tasks from 87.6% to 30% at 128 Attempts — iScienceLuvr · 2026-10-02
- [source] Meta paper: RL post-training hurts test-time scalability — the 'Sharpening Tax' — dair_ai · 2026-10-04