Qwen 3.8 Flash Next uses N-gram table to save backbone weights
antirez · x · 2026-08-30
antirez explains the N-gram mechanism in Qwen 3.8 Flash Next: at Layer 1, it takes the last two and three tokens to retrieve 16 small vectors from a 51B database. These are composed into a larger vector and gated to enrich the current token at Layer 2. This approach saves significant weights on the model backbone by offloading fixed associations like "New York" to the table.
More from Models
- Google's "ungrounded beliefs" classifier injections derail Opus 4.1's reasoning chain — liminal_bardo · 2026-10-16
- A week of ChatGPT Pro quota burned in 36 hours as GPT-6 fails to factor a 512-bit RSA key — matthew_d_green · 2026-10-16
- Gemini 4 Argon hits 29/36 proven DFIR verdicts, best in 22-model security benchmark — DynamicWebPaige · 2026-10-16
- Same debate, with chart: can harnesses sustain AI progress past the plateau? — Otherwise-Report1848 · 2026-10-16
- Mistral touts STEM benchmark win, but leaderboard shows it ranking sixth overall — MistralAI · 2026-10-16
- Local tool calling compared: how Gemma 4, Llama 3 and Mistral differ — leslysandra · 2026-10-16