Raschka: open LLMs have abandoned full attention — hybrids and linear models now split 50/50
rasbt · x · 2026-11-11
Sebastian Raschka shares a trend from his upcoming open LLM roundup: by late 2026, zero new open models use full attention throughout. Hybrids mixing global attention with sliding-window/Gated DeltaNet/Mamba-2/sparse attention and pure non-full-attention models now split 50/50, driven by cheaper long context for documents, large codebases, and tool-calling agents.
More from Models
- Ethereum dev sits on 4+ home automation vulns, forced to use Chinese models as Claude refuses — TheMoonMidas · 2026-11-11
- Rumored 'Gemini 4 Carbon' shown generating a portrait of Sundar Pichai — CodeByPoonam · 2026-11-11
- Gemini 4 Carbon tops Cursorbench coding benchmark, LuminaBench reports — melvinjohnsonp · 2026-11-11
- EU user says Gemini blocks access even on VPN inside supported EU countries — Darth_Ender_Ro · 2026-11-11
- BrickVision benchmark shows VLMs often ignore their own visual grounding — AndrewDai · 2026-11-11
- Singularity Gate: Fable 5.2 tops benchmark at 25.8%, but all models stall near 8% on the hardest tasks — queenofartists · 2026-11-11