Qwen3.6-35B on one RTX 3060 12GB: RustingStrata beats llama.cpp in new benchmark
Serious_Scarcity7818 · reddit · 2026-11-10
A Reddit user benchmarked three inference engines on a single RTX 3060 12GB (Ryzen 5 7600X, 32GB RAM) using the same Unsloth GGUF of Qwen3.6-35B-A3B (Q3KM, 15.9 GiB, spills past VRAM), 5290-token prompt, 128 generated tokens, mean of 3 runs:
- RustingStrata: 2330 tok/s prefill; 90.9 tok/s decode, 128.4 tok/s with MTP drafts
- PolyStrata: 1458-1549 tok/s prefill; 78.5 tok/s decode (no MTP), 100 tok/s with MTP
- llama.cpp b12119: 1692 tok/s prefill (-ncmoe 22); 53.5 tok/s decode, 77.7 tok/s with draft-mtp (92/102 drafts accepted)
The author documents all commands and caveats: PolyStrata only offers three quants and is built on llama.cpp CUDA kernels; MTP gains depend on text type; the no-MTP row is the clean RustingStrata-vs-llama.cpp comparison. Even so, a 3B-active MoE model hits 128 tok/s decode on a 12GB consumer card with the newer engine.
More from Infra
- Running open-weight models locally: Kimi K3.1 needs 594GB, ~$38k for 15 tok/s — ravikantagrawal · 2026-11-10
- InfraRSI Applies Recursive Self-Improvement to Training Infra: 2.75x Speedup, MFU Lifted from 17.4% to 30% — TokenRhythm · 2026-11-10
- ClickBench Playground Lets You Run SQL Against 110+ Databases Side by Side — mgill25 · 2026-11-10
- Musk: new Grok training stack derived from Starlink's in-house engineering — elonmusk · 2026-11-10
- 4x RTX 3090 vs one RTX 5090: Malaysian builder weighs VRAM against newer architecture — whity2773 · 2026-11-10
- Hybrid-Spark switches drafters per decoding step, up to 1.62x faster than DSpark on code edits — Julianfmack · 2026-11-10