Strata hits 3x decode and 26x prefill speedups over llama.cpp in local tests
relmny · reddit · 2026-10-13
Reddit user relmny benchmarked the Strata local inference engine against llama.cpp (Unsloth fork) on the same qwen3.8-flash-next-unsloth-ud-q4kxl model with fp16 KV and 130k context:
- Prefill: a 38,968-token prompt took 10 minutes in llama.cpp vs 23 seconds in Strata (26x faster)
- Decode: Strata sustains 57-58 tok/s vs llama.cpp's 18.8 tok/s (3x)
- Same model files were used; adding --kv-resident 32768 to llama.cpp didn't help much
After 10 days of daily use, results remain consistent and recent Strata updates pushed throughput further (previously 45 tok/s). Logs show a 94.5% expert cache hit rate, caching routed experts over PCIe.
More from Infra
- Ex-Apple engineer eyeing Surface with NVIDIA RTX Spark: 128GB + CUDA in a backpack — LinusEkenstam · 2026-10-13
- ML Veteran John Langford Endorses GPU MODE's Open Benchmark Competition Model — JohnCLangford · 2026-10-13
- Transcribing an hour of audio in ~13s on a MacBook Air — amrrs · 2026-10-13
- QuantWM: training-free 2-bit KV cache quantization cuts video world model memory 6x without flicker — jiqizhixin · 2026-10-13
- User buys two DGX Spark units at ~$6,100 each, finds local GLM 5.3 setup info scarce — MasterNomie · 2026-10-13
- Real-time voice agent model survey: teams cut LLM latency from 1s to 0.3s by switching off GPT-4.1 — Natural-Chemical1513 · 2026-10-13