RustingStrata hits 160 tok/s on RTX 3060 running 35B MoE, ~2x faster than llama.cpp
Serious_Scarcity7818 · reddit · 2026-11-08
Duplicate post of the same RustingStrata release: the pure-Rust engine decodes Qwen3.6-35B-A3B (Q2KXL, some experts on CPU) on a 12 GB RTX 3060 at 118 tok/s, and up to 160 tok/s with MTP speculative decoding — 2x llama.cpp (68 tok/s). MTP made llama.cpp slower and crashed on some settings. Prefill 2200 tok/s matches llama.cpp; 32K context does 1800 prefill / 92 decode; Vulkan matches CUDA speed; output matches top-1 98-99%.
Related event: Pure-Rust inference engine doubles llama.cpp speed on aging GPUs(2 posts)→
More from coding & agent
- Baseten Loops goes GA with serverless SFT/RL training, per-token pricing — baseten · 2026-11-08
- Software engineer asks how to bring AI agents into his family's retail business — FIREATWlLL · 2026-11-08
- Where do your agents run when the laptop closes? 4 hosting options compared — michaelTM_ai · 2026-11-08
- Where do browser agents actually break on the real web? Login walls, popups, or worse — lucy_yarntra007 · 2026-11-08
- Anthropic's 40-min AI-native SDLC playbook: code is no longer the bottleneck — blaizedsouza · 2026-11-08
- The software factory is the next extinct wrapper: coding agents will absorb orchestration value in 3-6 months — blaizedsouza · 2026-11-08