vLLM finally runs on 4x Intel B70 GPUs: 450 t/s prefill after overflow fixes
38andstillgoing · reddit · 2026-10-17
A user reports getting vLLM working on 4x Intel B70 GPUs running QFN 3.8 FP8, with 128GB VRAM plus 128GB RAM in a containerized setup.
- Measured performance: 450 t/s prefill and 20 t/s generation, below published benchmarks since multi-card support required extra surgery.
- Key fix: with help from LLMs (mostly GLM 5.3 flash), several 2^32 integer overflow bugs were found and fixed/backported—these had caused random crashes on multi-B70 setups.
- The fixes and reproducible steps are open-sourced on GitHub (wormley/B70Quad).
More from Infra
- Compute Labs launches financing platform matching GPU operators with capital — paigeinsf · 2026-10-17
- Google researcher explains why 5:1 interleaved SWA won: XLA compile times and TPU constraints — teortaxesTex · 2026-10-17
- Anthropic lands BB+ rating as Wall Street assembles $60B AI chip financing — basedjensen · 2026-10-17
- Qdrant tests Perplexity's PDF embeddings: 21× smaller vectors, 96.5% recall kept — jrxna · 2026-10-17
- Anthropic signs lease for its first Canadian data center in Nova Scotia — Polymarket · 2026-10-17
- HRT combines eBPF, service logs and hardware counters for byte-level network attribution — tianyin_xu · 2026-10-17