DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API
musicurgy · reddit · 2026-10-28
Reddit user musicurgy released delta-inference-engine, an MIT-licensed Windows-native inference engine (a port of Xolotl/Strata) that runs DeepSeek-V4.1-Flash-1017 Q2 GGUF on a single RTX 3090 with 192GB RAM at 41 tok/s, full 1M context, no WSL needed. Output matches DeepSeek's own API on the next token 90.2% for general text and 92.4% for code/math. Benchmarks cover nazeshinjite, ISTA-DASLab GSQ-RCO, and Unsloth quants (IQ1M up to 42 tok/s); needs 12GB+ VRAM (17–41 tok/s), 128GB+ RAM, and 385GB NVMe. One-click .bat setup starts an OpenAI-compatible localhost server with SHA-256 checks and no prompt logging. Author credits Claude for most of the coding.
More from coding & agent
- Company starts 'hiring' AI bots into its sales team for CRM, quotes and deal review — Baconbrix · 2026-10-28
- Vibe-coding paradox: you can now build any app you want — and still never have time to use it — bigblueboo · 2026-10-28
- CI demand surges 9x in a week as AI-generated code floods pipelines, Celesto shares scaling fixes — aniketmaurya · 2026-10-28
- Capsem: Rust hypervisor from DeepMind team runs concurrent OCI containers for isolated AI agents — rakyll · 2026-10-28
- Dev open-sources an MCP server that deterministically checks SQL before your agent runs it — Greedy_Ad8932 · 2026-10-28
- Offloading Claude Code subagents to DeepSeek: $2000/day down to $9 — cramforce · 2026-10-28