One Ranking, Any Budget: Elastic Transformer Prunes to 60% Sparsity Without Retraining
cgmsnoek · x · 2026-10-15
Researchers introduce an "elastic Transformer": a single importance ranking over all MLP neurons and attention heads, learned once with frozen weights, lets a model be pruned to any compute budget at inference. Across 27 vision and text encoders (22M–8B params), performance degrades smoothly up to 60% sparsity — e.g., DINOv3 ViT-H+/16 at 50% sparsity drops 427M parameters for only a 0.8% accuracy loss.
Related event: Elastic Transformers: Training-Free Pruning at Any Compute Budget(2 posts)→
More from Research
- Microsoft's Magnetar zkVM proves programs 10-39x faster using only standard hash functions — Microsoft Research · 2026-10-16
- Fine-tuned Nemotron 3.5 lifts hard Verus proof pass rate from 43% to 73% — fhuszar · 2026-10-15
- AI turns numerical-analysis intuition into Lean proofs with error bounds near 1e-48 — geoffreyirving · 2026-10-15
- Ex-UK AISI chief scientist puts odds of superintelligence killing everyone at a coin flip — geoffreyirving · 2026-10-15
- There is no posttraining data commons: open models' next data bottleneck — dlwh · 2026-10-15
- tszzl: agentic auto-research makes mechinterp finally reveal what's inside models' minds — tszzl · 2026-10-15