Open PR optimizes DeepSeek-V4.1 inference on RTX PRO 6000 Blackwell
AlpinDale · x · 2026-10-22
Developer AlpinDale submitted PR #1788 to dphnAI/sonar, optimizing DeepSeek-V4.1-Flash inference on RTX PRO 6000 Blackwell (SM120) for both single-stream and batched decoding.
Key changes: context-bounded persistent DSA indexing, native DeepGEMM support for 128-token indexer pages, hierarchical top-k selection, and SM120 fast paths for dense MXFP8 projections, grouped output projections, mHC, and Engram pack/finish. It also fixes shared-KV scale offsets, preserves tensor-view offsets through the weight-cache path, and handles masked top-k tails.
Benchmarked on 4× RTX PRO 6000 Blackwell 96GB with TP4, B12x 1.5.1, FP8 KV cache, CPU-offloaded Engram, and compilation/CUDA graphs enabled.
More from Infra
- Ollama Runs Open Models on NVIDIA Blackwell via Crusoe, Live in Two Weeks — ollama · 2026-10-22
- Claim: every 2nd/3rd-tier inference provider now has a recipe for 90%+ compute margins — zephyr_z9 · 2026-10-22
- NVIDIA keynote: silicon photonics CPO switches hit 102.4 Tb/s, Vera Rubin architecture unveiled — thealexbanks · 2026-10-22
- PyTorch Foundation Adds Nine Members, Nebius, Nscale and Shopify Join as Platinum — PyTorch · 2026-10-22
- GPU startup Flappy Airplanes raises $1B Series A co-led by Index and Lux — vitaliychiley · 2026-10-22
- Crusoe unveils Tinker API-compatible LLM post-training platform at PyTorch Conference — PyTorch · 2026-10-22