RustingStrata Benchmarked: 29% Faster Than llama.cpp With Near-Identical Output Quality
Serious_Scarcity7818 · reddit · 2026-11-11
RustingStrata, a pure-Rust MoE inference engine from RustingStudio for running Qwen GGUF models on 12 GB consumer GPUs (claims 2× llama.cpp prefill on an RTX 3060), was independently tested against quality-loss accusations:
- Speed: same Qwen IQ2XS file, prompt, and 128-token reply on RTX 3060 / Ryzen 7600X, six warm runs each — 33.0 tok/s generation (29.3–35.8) vs llama.cpp's 25.6 tok/s (23.2–26.9), 29% faster; first-prompt read 109 tok/s vs 31 tok/s
- Quality: over 9,180 WikiText-2 tokens, perplexity 2.9212 vs 2.9156 (noise-level gap), 96.78% identical top tokens, mean KLD 0.0098
The author caveats this is one model, one quantization, one card — not a universal win — and links the repo for reproduction.
More from Infra
- DeepSeek V4.1-Flash seeds resist 4.3B-seed search; Huawei chip shift rumored — TheNormanMu · 2026-11-13
- OpenCartridge's trust bill: no auth, public bucket, Docker socket exposed — maier_ak · 2026-11-13
- camenduru open-sources OpenCartridge, packaging AI models as one-click 'cartridges' — maier_ak · 2026-11-13
- Game assets on your own GPU: what OpenCartridge does and what it asks you to trust — maier_ak · 2026-11-13
- Core Memory podcast: Sceye's $100M high-altitude airship wins SoftBank fleet order for Japan broadband — kyliebytes · 2026-11-13
- Torus explains closed-loop liquid cooling: no on-site water hookup needed — ccerrato147 · 2026-11-13