IQuest-Q1 Specs Leak: 524K Context, 256 Experts, Hybrid Attention
jacek2023 · reddit · 2026-09-29
Detailed specs of IQuest-Q1 (MoE, 320B total / 15B active) surfaced on Reddit: 88 transformer layers, hidden dim 3072, 48/8 Q/KV heads, a 3 SWA + 1 full-attention hybrid pattern with 4096 sliding window, 256 experts with 8 activated per token, multi-token prediction (2 independent layers in training, 1 recursive x8 at inference), and a 524,288-token context length.
Related event: IQuest Open-Sources IQuest-Q1, a 320B MoE Model for Agentic Coding(11 posts)→
More from Models
- Fable 5.2 Leads MazeBench Token Efficiency; Billion-Token Runs Underway — patience_cave · 2026-11-16
- Fable 5.2's Puzzle Skills Look Suspiciously Good, Reciting Outdated MazeBench Solution — patience_cave · 2026-11-16
- Fable 5.2 Tops MazeBench With 21% After 90 Hours and $6,000 in Credits — patience_cave · 2026-11-16
- Tencent Hy4 flash and InternLM Sage-S2.1-Flash both released, tool-calling tests requested — MajesticPuzzler · 2026-11-16
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16