Google Proposes Muir, a Linear Attention Architecture Beating Gated DeltaNet-2 and Transformers
google · hf · 2026-11-07
Google published Muir, a linear attention architecture that allocates gate capacity by sensitivity, uses matched-filter readout and a stable parallel update. It improves language modeling, long-context recall and training speed over Gated DeltaNet-2 and Transformer baselines. An accompanying needle-in-haystack chart shows Muir's equalized readout reliably recovering a needle planted at token 600 after streaming 8,000 tokens, where GDN-2's standard readout shows uneven per-layer signal.
More from Research
- GLASS preprint teaser: graph-led LLM-assisted scoring of open-ended responses without human coding — GolinoHudson · 2026-11-07
- ICLR 2027 reviews mocked as worse than the papers, as LLMs improve mid-cycle — miniapeur · 2026-11-07
- LLMs can read and patch weights, but 20 fine-tuning steps erase the edits — kevinpl07 · 2026-11-07
- FlexiTac-Gen launches: AI-agent-assisted tool to design custom tactile sensors, accepted to CoRL 2026 — Haoyu_Xiong_ · 2026-11-07
- PolicyIK: NYU trains RL for inverse kinematics on a real arm, hitting 0.5mm median error — Lianhuiq · 2026-11-07
- Weight-space programming vs LoRA on 4 bugs: precise one-neuron patches that don't persist — kevinpl07 · 2026-11-07