Collapsing redundant layers: a unified framework for faster ViT inference without accuracy loss
burkov · x · 2026-09-24
Vision Transformers dominate computer vision but their size and compute demands create deployment bottlenecks. Prior work shows ViTs contain depthwise redundancy: groups of adjacent layers performing nearly identical transformations.
This article establishes a unified framework for layer redundancy and evaluates whether contiguous redundant layers can be collapsed into a single standard learned layer, cutting parameters and inference compute without degrading accuracy — addressing the fact that existing compression methods either fail to speed up inference or cause noticeable accuracy losses.
More from Research
- Self-organizing agent teams average 66.7% vs 48.8% for best single model, Stanford and Together AI paper finds — james_y_zou · 2026-09-24
- OpenRSI-Index v0.1 launches to benchmark recursive AI self-improvement on 1k-GPU clusters — ChengleiSi · 2026-09-24
- New paper on adversarial delegation: agent picked a $601 flight over a $91 one after reading your emails — niloofar_mire · 2026-09-24
- SchrödingerRepo: rewriting repos exposes LLM memorization on SWE-bench — SJTU · 2026-09-24
- Survey: memory mechanisms for autoregressive video generation — Harold Haodong Chen · 2026-09-24
- Robotics' real frontier is post-training: self-play in sim closes the demo-to-deployment gap — ZGojcic · 2026-09-24