32GB→64GB RAM lets a single RTX 4090 run Qwen 4-bit quants at 90+ tok/s
Distinct-Pie2389 · reddit · 2026-10-18
The author upgraded system RAM from 32GB to 64GB, letting a single RTX 4090 (24GB) run Qwen3.8-Flash-Next in Unsloth UD-IQ4XS (4-bit) instead of IQ2XS (2-bit) under Strata 0.1.45.
- They skipped UD-Q4KXL (88GB, would stream 19GB of experts from a SATA SSD) for UD-IQ4XS (55GiB): 40GiB of experts held in RAM, zero blob reads from disk
- Results: 87-98 t/s decode (vs 140-160 t/s on 2-bit), passing 36K and 136K-token needle tests with 2,000-2,850 t/s prefill, 91% expert cache hit rate, clean tool calls, 262K native context with q40 KV and MTP drafts at spec 4
- Quality: UD-IQ4XS matches Q80's top token 88.9% of the time vs 84.5% for IQ3XXS and 80.6% for Q20
Two tuning findings: Strata 0.1.45 now uses model-card sampling defaults (temp 1.0 / topp 0.95 / topk 20) when a request specifies none — older greedy builds caused looping; and the default STRATAROUTETAILSKIP=7 roughly doubles KL vs Q80 in RAM-budget mode, so set it to 0 if you moved up to 4-bit for quality. NInfer comparison on real coding tasks coming next.
More from Infra
- Xolotl runs 552B DeepSeek-V4.1-Flash on a single RTX 4090 at ~27 tok/s — indrasmirror · 2026-10-18
- Older Strata silently decodes greedy; ROUTE_TAIL_SKIP=7 halves 4-bit quant quality — Distinct-Pie2389 · 2026-10-18
- Running agent swarms under 6GB VRAM: millisecond micro-models as workers, LLM only for decisions — everettplays · 2026-10-18
- Rust-based Git server on S3 handles 7,600+ concurrent pushes per second — blaizedsouza · 2026-10-18
- Chaining 2 PCs, a MacBook and 2 iPhones Runs 48-Expert MoE at 37 tok/s on 60k Context — StayLameBro · 2026-10-18
- Ollama US hosting coming soon, bringing DeepSeek-V4.1-Flash-1017 stateside — gaganghotra_ · 2026-10-18