Strata engine underwhelms on 2× R9700: loses to llama.cpp despite higher t/s
Don_Reuter · reddit · 2026-11-01
A Reddit user tested the Strata inference engine on dual R9700 cards (64GB VRAM), hoping to match Qwen 3.8 27B Q8 + MTP on llama.cpp with larger context. Results were disappointing: offloading experts to RAM hangs on long tasks in every config; it's only stable with all experts in VRAM, forcing a 3-bit quant. It benchmarks 60 t/s vs llama.cpp, but on HumanEval+ the 3-bit Flash-Next scored 93.9% vs 95.1% for Q8 on plain llama.cpp, while generating nearly twice the tokens. The author concludes custom model-specific engines bring forks, special quants and env-var workarounds without real-world gains, and asks whether the hype posts are edge cases.
More from Infra
- Where the money goes per token: a diagram of LLM API cost anatomy — cneuralnetwork · 2026-11-01
- FreeToken brings datacenter-scale MoE model serving to desktops, hits 15k stars — FlashML-org · 2026-11-01
- Musk claims xAI's capex efficiency is greatly underestimated — RachelVT42 · 2026-11-01
- AI bubble pops globally: China wipes out ¥23.5T, Nasdaq drops 3.46% in a day — Grady_Booch · 2026-11-01
- exo clusters your Macs into one AI machine to run frontier models locally — alex_verem · 2026-11-01
- 6 open source tools to run AI locally on your laptop, no cloud required — alex_verem · 2026-11-01