Qwen 3.8 Flash Next uses N-gram table to save backbone weights
antirez · x · 2026-08-30
antirez explains the N-gram mechanism in Qwen 3.8 Flash Next: at Layer 1, it takes the last two and three tokens to retrieve 16 small vectors from a 51B database. These are composed into a larger vector and gated to enrich the current token at Layer 2. This approach saves significant weights on the model backbone by offloading fixed associations like "New York" to the table.
More from Models
- Leak: Google Gemini 3.8 Flash coming soon with major quality boost — mark_k · 2026-08-30
- GLM-5.3-Flash generates games in real-time on Mac Studio — MaziyarPanahi · 2026-08-30
- GPT Astra takes 38 minutes for 56k tokens: token efficiency is not compute efficiency — ___Patrice___ · 2026-08-30
- GLM-5.3-Flash Beats GPT-5 in Coding Arena at 26x Lower Cost — ccerrato147 · 2026-08-30
- Opus 5 criticized for jargon; author highlights LLMs' struggle to explain simple concepts clearly — antirez · 2026-08-30
- LongCat-Flash-Lite-Sparse and Qwen Uncensored Models Released in GGUF — LLMFan46 · 2026-08-30