New PCIe-optimized all-reduce kernels boost DeepSeek decode 21% (32% with spec decode)
teortaxesTex · x · 2026-10-26
Developer AlpinDale released all-reduce kernels optimizing small-message latency across PCIe-connected GPUs, delivering a 12% end-to-end decode speedup for DeepSeek-V4.1 Flash on a 4x RTX 6000 Pro server. After further tweaks, results improved to 21% without speculative decoding and 32% with it, averaged over 10 sequential requests on 3 separate servers to minimize variance. Bench data: non-speculative decode rises from 101.2 tok/s (NCCL) to 122-123 tok/s (paired PCIe); DSpark K7 speculative mode jumps from 290.4 to 385.8 tok/s with 60% draft acceptance.
More from Infra
- Meta files $35B stock and mandatory convertible offering, plus $60B ATM program — RihardJarc · 2026-10-26
- "The grid and your home are the bottleneck": compute buyer goes bullish on ARM chips — TheZachMueller · 2026-10-26
- On-device VLM goes production: GenieX runs Liquid AI's LFM-VL live at PyTorch Conference — PyTorch · 2026-10-26
- a16z: OpenRouter token demand up ~27x in a year, open-weight models now serve 72% — TheZachMueller · 2026-10-26
- Giraffe: Open-Source Baseline Testing for OpenAI-Compatible LLM Endpoints — ImmediateBig8479 · 2026-10-26
- Local LLM hobbyist's self-roast Q&A goes viral: does it pay off? Only NVIDIA profits — TheZachMueller · 2026-10-26