Xolotl runs 552B DeepSeek-V4.1-Flash on a single RTX 4090 at ~27 tok/s
indrasmirror · reddit · 2026-10-18
Developer Indras-Mirror released Xolotl, a Strata-based inference engine that runs the full 552B-parameter DeepSeek-V4.1-Flash on one gaming PC:
- Hardware: single RTX 4090 (24GB), Ryzen 7 5700X, 90GB DDR4, NVMe — nothing exotic
- How: weights spread across VRAM, pinned RAM, and NVMe; the engine pulls in only the experts each token actually needs
- Results: 27 tok/s decode, 180-320 tok/s prefill, 512K context; 21/21 HumanEval, 10/10 tool calls, 3/3 agentic bug-fix loops with thinking on
- Same engine also runs GLM-5.3-Flash, Qwen3.8-Flash-Next, and Qwen3.8-27B
A 152GiB uncensored GSQ quant is already on Hugging Face; the author is downloading the new 510GB DeepSeek-V4.1-Flash-1017 (identical config/tokenizer) and expects a re-calibrated quant in a few days. Open source: github.com/Indras-Mirror/Xolotl.
More from Infra
- Running Qwen Flash Next on a Mac Mini M5 64GB: SSD expert streaming benchmarked with charts — turtleninja99 · 2026-10-18
- Older Strata silently decodes greedy; ROUTE_TAIL_SKIP=7 halves 4-bit quant quality — Distinct-Pie2389 · 2026-10-18
- Running agent swarms under 6GB VRAM: millisecond micro-models as workers, LLM only for decisions — everettplays · 2026-10-18
- Rust-based Git server on S3 handles 7,600+ concurrent pushes per second — blaizedsouza · 2026-10-18
- Chaining 2 PCs, a MacBook and 2 iPhones Runs 48-Expert MoE at 37 tok/s on 60k Context — StayLameBro · 2026-10-18
- Ollama US hosting coming soon, bringing DeepSeek-V4.1-Flash-1017 stateside — gaganghotra_ · 2026-10-18