Running Strata locally on dual 3090s: IQ3_S hits ~100 tok/s, IQ4_XS drops to ~30
klasyer · reddit · 2026-10-28
A Reddit user shares real numbers from running Strata locally on dual RTX 3090s (3900X, 128GB RAM): Unsloth IQ3S outputs 100 tok/s, IQ4XS 30 tok/s, Q4KXL under 20 tok/s, preferring IQ4XS for fewer hallucinations. Setup is a Docker container with 256K context, both GPUs, q40 KV cache, 10-minute idle unload, and THOROUGH tuning — full docker-compose YAML included — asking the community for further optimization tips.
More from Infra
- GPU Neoclouds Run on Airline Economics, Argues Ex-ABS Trader Alex Song — BenBajarin · 2026-10-28
- XPeng's IRON solid-state battery enters small-batch production: 30% lighter, 30% more capacity — CyberRobooo · 2026-10-28
- Visualizing All fp32 Bit Patterns to Reveal How Fast-Math Unary Ops Diverge — HanGuo97 · 2026-10-28
- PC downturn frees wafers but not the AI bottlenecks: packaging and power cap accelerators — BenBajarin · 2026-10-28
- Wan2.2 TI2V-5B on a single RTX 4090: ~8.5 min per 5s clip, faces broke in 2 of 3 tests — NoEagle3604 · 2026-10-28
- Cloud Providers Can't Forget What Enterprises Actually Run On — DavidLinthicum · 2026-10-28