LayerTrim: training-free pruning keeps reasoning LLMs intact on AIME
SeoulNatlUniv · hf · 2026-10-16
Researchers from Seoul National University, NYU, and USC propose LayerTrim, a training-free structured pruning method for reasoning language models, addressing why existing pruning collapses performance on long-form reasoning tasks like AIME.
- Method: identifies redundant decoder layers, removes them, and selectively trims their MLP features while preserving attention and decoder depth.
- Results: cuts inference cost and generation length; the accuracy-vs-tokens plot shows LayerTrim far outperforming LaCo, GRASP, ShortGPT, FLAP, and RESP, approaching the Dense baseline.
- Significance: first training-free structured pruning designed to preserve attention for reasoning models; code is open-sourced on GitHub.
More from Research
- VPO: The Post-Training Method That Teaches Models to Output Diverse Good Answers — lateinteraction · 2026-10-16
- 797GB of n-grams in read-only CPU RAM match 57% more compute at 1B-scale LM pretraining — anshulkundaje · 2026-10-16
- New essay: prioritizing falsifiability in AI cognitive modeling can be reward-hacked — AndrewLampinen · 2026-10-16
- SPLICE: repairing failed math reasoning with solution hints boosts AIME and proofs — behrouz_ali · 2026-10-16
- JAMA launches JAMA AI, a new journal for clinically focused AI research — pshrink · 2026-10-16
- Retrofitting pretrained transformers with Titan modules speeds up inference — behrouz_ali · 2026-10-16