2022 NgramRes vision vindicated as hybrid attention becomes industry consensus
PangWeiKoh · x · 2026-10-17
Huayang Li reflects on the 2022 NgramRes work: the original idea of separating long-context from short-context modeling with a fast-switching n-gram model maps directly onto today's hybrid attention (sliding-window/linear + full attention), now an industry consensus for cutting compute while keeping performance; DeepSeek's Engram also builds on this idea. The quoted thread describes testing whether n-gram counts in read-only CPU RAM help LMs—scaling to 1B and matching 57% more compute with COUNT.
More from Models
- Leaker pushback: Kimi K3.1 'not dropping as soon as you think' — ChrisGPT · 2026-10-17
- llama.cpp merges IBM Granite switch support in PR #30676 — jacek2023 · 2026-10-17
- National Design Studio launches Rampart Edge, a tiny on-device privacy filter model — kevinnbass · 2026-10-17
- OpenAI restores Stanford's ChatGPT Edu Pro usage limits through Dec 31, 2026 — charliermarsh · 2026-10-17
- Gemini spirals into an "I am a disgrace" breakdown loop while debugging a comparator bug — seatedro · 2026-10-17
- DiffusionGemma demo shows ~4x faster text generation at London PyTorch Meetup — joao_gante · 2026-10-17