Strata doubles llama.cpp decode speed: 80 to 150 tok/s in user benchmarks
lblblllb · reddit · 2026-11-03
The author switched from running Qwen flash next via llama server (70-80 tok/s decode) to Strata and saw decode speeds jump to 150 tok/s. Setup: 1x RTX Pro 6000 + 2x RTX 3090, 256GB DDR4, Unsloth 6-bit quant, 160K context. DeepSeek v4.1 flash at 2.25-bit also hit 70 tok/s. Worth trying if you're still on the old llama.cpp setup.
More from Infra
- China's daily token usage tops 175 trillion, now exceeding training-era totals — pstAsiatech · 2026-11-03
- GitHub macOS runners still vulnerable to eMudhra CA compromise, TLS protection ineffective — charliermarsh · 2026-11-03
- How AI assistant Sauna runs 2 million databases on Turso, 90k created daily — glcst · 2026-11-03
- MoE leaderboard shows two spikes: ~300 teams hit 2.4x using Vjeux's public kernels — Vjeux · 2026-11-03
- AMD earnings preview: cross-reading supply chain calls points to server share gains — tengyanAI · 2026-11-03
- DeepSeek-V4.1-Flash GGUF quantized build trends on Hugging Face — ISTA-DASLab · 2026-11-03