Researchers say latent feedback should read mid-layers, not the last
HKU MMLab and Yale researchers scanned feedback sources in Feedback Transformers and found reading the second-to-last layer outperforms the common last-layer approach used in latent feedback methods like FBT and Recycler.
2026-11-18 ~ 2026-11-18 · 2 related posts
- Latent feedback debate: read a deep intermediate block, not the last — lower loss at same FLOPs — kastnerkyle · 2026-11-18
- Feedback transformers work better reading below the top block, HKU study finds — JohnCLangford · 2026-11-18