FULL STORY
RustingStrata: Rust Inference Engine Launches, Beats llama.cpp in Tests
RustingStrata, a pure Rust GGUF inference engine for small-VRAM setups, launched in early November; community benchmarks soon showed it running about 29% faster than llama.cpp with negligible quality loss.
2026-11-08 ~ 2026-11-11 · 2 episodes · 6 posts
Episode 1 · Rust-Only Inference Engine RustingStrata Doubles llama.cpp Speed on Consumer GPU (2026-11-08, 4 posts)
RustingStrata, a pure Rust GGUF inference engine with CUDA/Vulkan backends, runs Qwen3.6-35B-A3B at 160 tok/s on a 12GB RTX 3060 with partial CPU offloading, twice as fast as llama.cpp, and can even fit a 249GB model into 12GB VRAM plus 30GB RAM.
- RustingStrata hits 160 tok/s on RTX 3060 running 35B MoE, ~2x faster than llama.cpp — Serious_Scarcity7818 · 2026-11-08
- RustingStrata hits 160 tok/s on RTX 3060 running 35B MoE, ~2x faster than llama.cpp — Serious_Scarcity7818 · 2026-11-08
- Pure-Rust Inference Engine RustingStrata Doubles llama.cpp Prefill Speed for Qwen3.6-35B on a 12GB GPU — Serious_Scarcity7818 · 2026-11-09
- Pure-Rust engine runs 249GB DeepSeek model on a single 12GB RTX 3060 — Serious_Scarcity7818 · 2026-11-10
Episode 2 · RustingStrata Benchmarks: 29% Faster Than llama.cpp With Negligible Quality Loss (2026-11-11, 2 posts)
Benchmarks show RustingStrata, a pure-Rust MoE inference engine, runs Qwen quantized models about 29% faster than llama.cpp on an RTX 3060 12GB with quality loss within noise.
- RustingStrata benchmarked 1.29x faster than llama.cpp with no measurable quality loss on RTX 3060 — Serious_Scarcity7818 · 2026-11-11
- RustingStrata Benchmarked: 29% Faster Than llama.cpp With Near-Identical Output Quality — Serious_Scarcity7818 · 2026-11-11