Microsoft Research Proposes Full-Bandwidth Transformer: Latent Feedback Boosts Performance for Free
YouJiacheng · x · 2026-08-11
Microsoft AI Frontiers researchers propose the full-bandwidth transformer, which feeds the previous top-layer hidden state back into the input via a gated linear unit, widening the vertical feedback channel. Trained with a scheduled multi-pass objective, 1B-parameter models show improved validation loss, language modeling, math, and code generation, with negligible decoding overhead, matching standard transformers trained with 1.5x more tokens.
More from Research
- Tencent's T-Mem fixes similarity-retrieval blind spot in AI agent memory, EMNLP 2026 — jiqizhixin · 2026-09-22
- JEVfire open-sourced: Qwen 0.8B clears Super Mario in-browser at 71ms per action — ricklamers · 2026-09-22
- RegVGGT: training-free token regulation keeps only 1% of tokens per frame for streaming 3D reconstruction — zhenjun_zhao · 2026-09-22
- D3GS: depth, DINO and diffusion co-guided 3D Gaussian Splatting for sparse-view reconstruction — zhenjun_zhao · 2026-09-22
- VGGT-Prime: compute-adaptive mixture-of-heads slashes redundancy in visual geometry transformers — zhenjun_zhao · 2026-09-22
- Elevator-VIGS: Gaussian Splatting SLAM that keeps tracking through elevator rides, zero-shot via VLM — zhenjun_zhao · 2026-09-22