Fixed-Point Self-Attention (FPSA) accepted to NeurIPS 2026: iterative context refinement with zero extra parameters
PlisSergey · x · 2026-09-26
Researchers announced that Fixed-Point Self-Attention (FPSA) has been accepted to NeurIPS 2026. FPSA turns attention into an in-layer dynamical system that iteratively refines context to an equilibrium (fixed point) without adding parameters. The authors frame it as a step toward models that "ruminate" to counterbalance their size. A detailed technical blog and code are forthcoming. Collaborators include Sergey Plis and BAPearlmutter.
More from Research
- Japan's Tooth-Regrowth Drug Blocking USAG-1 Completes Phase I, Kids Trial Next — bumpthebass · 2026-09-26
- Eval answer keys pick central examples while real answers are quirky, dev argues — voooooogel · 2026-09-26
- Quote arguing autoregressive error critique conflates prefix with full compute state — teortaxesTex · 2026-09-26
- New Paper Links Orch OR Theory With Microtubule Resonance — anirbanbandyo · 2026-09-26
- 4B decision model Mica beats same-size rival at Tetris without generating a single token — Top-Evidence174 · 2026-09-26
- Physicists scooped by Anthropic AI: 'more low-hanging fruit than experts expect' — soumitrashukla9 · 2026-09-26