DeepSeek-V4.1-Flash runs on one RTX 3090 at 41 tok/s with Delta Inference Engine v0.93
musicurgy · reddit · 2026-11-06
Developer musicurgy shipped Delta Inference Engine v0.93, running DeepSeek-V4.1-Flash quantized natively on Windows with a single RTX 3090.
Key results:
- Unsloth IQ2 quant holds 41 tok/s at 192GB RAM and scores 57/99 on the author's hard-question benchmark — within 2 points of the full API model (59), while using 11% fewer tokens
- RAM floor drops sharply: 64/96/128GB configs now usable (IQ1S at 30 tok/s on 64GB), with SSD read-ahead adding 22% at 64GB
- First token 40% sooner (3.8s → 2.3s on 180-token prompts); long prompts read at up to 740 tok/s
- Next-token agreement with DeepSeek API: 89–97%; author notes two full-model hosts only agree 98.3–98.9%, so IQ2 is near the ceiling
- New: native Unsloth Engram table reader, EXL3, basic multi-GPU (7% with 3060+3090), public CI builds, fixes past 65,535-token contexts
Open source on GitHub as delta-inference-engine.
More from Infra
- SpaceX data center pledges 40% power discounts and billions in community benefits — Scobleizer · 2026-11-06
- Mystery seller seeks to dump $500 million of data center loans — dinabass · 2026-11-06
- Collinear AI runs full-scale agentic RL on Together Post-Training with self-onboarding — togethercompute · 2026-11-06
- SemiAnalysis: Google TPUv7 Delivers Up to 50% Better Performance-per-Dollar Than B200/B300 for Inference — vllm_project · 2026-11-06
- Meta President: new US data center build-outs will be 'very challenging' — SumitGup · 2026-11-06
- Veteran investor: memory stocks aren't cheap even at P/E of 3 as LTAs are meaningless — bookwormengr · 2026-11-06