Forward-only LLM adaptation cuts memory and speeds up inference without backprop

机器之心 · wechat · 2026-11-20

Researchers from Lund University and Google DeepMind presented a NeurIPS 2026 paper on adapting LLMs using only forward passes. The method exploits differences in angular concentration of activations across SVD components to identify which weight matrices to low-rank, eliminating backpropagation entirely.

Compared to gradient-based LASER-Grad and exhaustive LASER search, the forward-only search achieves lower peak memory, higher speedups, and comparable accuracy across tasks like knowledge reasoning, hallucination detection, and fact verification. Experiments span GPT-J, LLaMA2, DeepSeek V1, Mistral, Qwen2 (including MoE variants) and multimodal Gemma4 up to 57B parameters; ablations show attention weight matrices can be skipped to further speed up the search.

The authors argue that with no gradient computation, low memory footprint and high speedup, the approach could enable real-time online adaptation during inference, letting systems adjust models to specific tasks or user groups at low cost. Paper: arxiv.org/abs/2611.10610.

Original post →

More from Research

Research channel →