Benchmark: Speedy inference engine crushes Avalon — 1.6x faster reads on Qwen3 long context
MotorNetwork380 · reddit · 2026-11-02
A local-LM user benchmarked Strata, Avalon, and Speedy inference engines on a 4090 + 128GB DDR5 + Ryzen 9 7950x rig, running IQ3S-quantized Qwen3 at 60K/120K/198K context.
Key numbers:
- Reading throughput: Speedy hits 4,400-4,600 tok/s — 1.6x Avalon, over 4x Strata
- Generation speed: all three similar (100 tok/s), Speedy 1-7% ahead
- Full-exchange wall time at 198K context: Speedy is 72% faster than Strata, 31% faster than Avalon
Verdict: having just set up Avalon, the author found Speedy "crushed it." Repo: github.com/DevDock-AI/Speedy.
More from Infra
- Crusoe CEO says neocloud debt fears overblown amid 'tidal wave' of compute demand — nathanbenaich · 2026-11-02
- Kapitsa hits 346 tok/s on a 35B MoE and runs a 180B model on 48GB Macs via SSD offloading — DotaMate · 2026-11-02
- Texas Mega-Project Pipeline Tops $500B, Led by $100B Stargate AI Hub — MatthewChang · 2026-11-02
- SemiAnalysis: 30.3GW of planned 2028 US data center capacity at risk of pause after midterms — Eliv_nurotic · 2026-11-02
- B300 GPU rental prices plunge from $7.9/hr to $4.6/hr in a month as providers beg for customers — SumitGup · 2026-11-02
- FT chart: Big Tech's surging CapEx is a rational bet, not reckless spending — luisdans · 2026-11-02