Saber, a Rust Inference Engine, Hits 6,645 tok/s Prefill on Qwen 27B
whodoneit1 · reddit · 2026-11-07
Yet another inference engine, Saber: written in Rust, supporting TP 1/2/4 and currently Qwen 3.8 27B plus Flash Next. The author ran it for over a month before deciding to release it, taking a different approach from existing engines. BetterBench results show Saber beats available engines on both the 27B and Flash Next, hitting 6,645 tok/s prefill and 316 tok/s decode peak on Qwen 3.8 27B. Open source at GitHub (GGZ14/saber-engine).
More from Infra
- Intel CEO: Agentic workloads could 5x server CPU TAM to $200B in 5 years — Beth_Kindig · 2026-11-07
- Personal AI agent rig: local 40 TOPS Arduino board + Tailscale/Cloudflare/Azure — unixterminal · 2026-11-07
- Big Tech capex up 5-8x since Q1 2023 while Apple spends 20% less; only Mag7 name beating Nasdaq — a16z · 2026-11-07
- Prefill-dominated endpoints coming for extraction and summarization workloads, researcher predicts — sh_reya · 2026-11-07
- SurrealDB 4 ships as 'the database for every agent' with PostgreSQL/MySQL support — mattturck · 2026-11-07
- India pledges indigenous GPU in 24 months; analyst says EUV access is the real clock — SumitGup · 2026-11-07