RustingStrata vs Strata on RTX 3060: Rust MoE engine edges out 28.3 vs 26.1 tok/s
Serious_Scarcity7818 · reddit · 2026-11-19
The author benchmarked his pure-Rust GGUF engine RustingStrata against Strata v0.2.13.1 running Qwen3.8-Flash-Next (experts split between VRAM and SSD) on an RTX 3060 12GB:
- Generation: 28.3 vs 26.1 tok/s with MTP drafting; first-token wait 5.4s vs 6.9s; repeated requests 36.9 vs 28.5 tok/s
- Draft acceptance 82% vs 69%, but drafting only adds 6% here — CPU-resident experts dominate the time
- Cold-start prefill: Strata is 2x faster (100 vs 45 tok/s)
RustingStrata is a general engine: one Rust binary, no Python/C++ build, CUDA kernels compiled on first start, same kernels on Vulkan (AMD/old NVIDIA), broader model support (Qwen3, GLM-5.3-Flash, Gemma 4, etc.), IQ1S–Q80 quants, vision input, runtime LoRA, dual-GPU layer split, OpenAI/Anthropic APIs and an MCP-capable chat page. Full launch commands for both engines are included.
More from Infra
- vLLM's DSpark for Kimi K3.1 Pro: 30% longer acceptance, bounded draft KV at 1M context — AccBalanced · 2026-11-19
- Costco sees panic buying of generators after string of tech outages, 21 of 26 sold out — harris_edouard · 2026-11-19
- Starlink direct-to-cell goes live in Mongolia with free first month — XFreeze · 2026-11-19
- SpaceX unveils $50B, 6GW Arizona compute campus with 24 GWh grid-battery strategy — JOBhakdi · 2026-11-19
- CoreWeave corrects debt rate to 15.5% as analyst presses Blackwell-funded payoff plans — Svwang1 · 2026-11-19
- tinygrad lists $89,000 tinybox with 768GB VRAM: four refurbished AMD MI300X GPUs — viksit · 2026-11-19