RTX 3070 runs quantized Qwen3.8 at 1000 tps prefill with Strata — full config shared
SetNo7692 · reddit · 2026-10-24
A Reddit user shared configs and benchmarks for running Qwen3.8-FN IQ3S via the Strata engine on modest hardware (Ryzen 5 5600, 64GB RAM, a €100 used RTX 3070): 1000 tps prefill, 40 tps decode, 262K context, int8 KV cache, and vision support.
Key optimizations:
- STRATAHCQ8: quantizes hyper-connection matrices to free VRAM for experts (slight KLD cost); one of the biggest prefill wins
- CPU-shared prefill computation
- Custom q8 n-gram table instead of the bundled iq4nl; speculative decoding set to 4
- gpu-unlock tool only added 50-100 tps prefill; 3060 owners may gain more
vs. the vanilla config that's roughly +500 tps prefill. Full JSON config is published in the post.
More from Infra
- Video DiT runs on 12GB VRAM at same speed via int8, model weighs only 9GB — cocktailpeanut · 2026-10-24
- Modal and Sakana AI optimize Namazu API to the kernel: NVFP4 quantization, DFlash drafting, KV routing — blelbach · 2026-10-24
- Semis debate: is high-NA EUV (EXE:5200) cost-effective for 18A nodes as TSMC demands tool upgrades — pstAsiatech · 2026-10-24
- Intel B60/B65/B70 GPU prices jump 30% in a day, B70 heading to $1,840 — SamSausages · 2026-10-24
- RTX 30-series GPU Unlock tested: up to 59% throughput gain in ComfyUI on 3090 Ti — under-_-null · 2026-10-24
- Cloudflare cuts CPU use 30% on D1 backend with new profiler, Workers Observability drops memory/CPU ~50% — irvinebroque · 2026-10-24