928-Star Wiki Details Running Qwen3.5-397B and Kimi-K2.5 on NVLink-Free PCIe RTX 6000
TheZachMueller · x · 2026-09-07
A community-maintained "RTX PRO 6000 Blackwell LLM Wiki" is trending on GitHub (928 stars), focused on serving frontier LLMs like Qwen3.5-397B, Kimi-K2.5, and GLM-5 on NVLink-free PCIe GPUs (SM120).
It goes well beyond launch snippets, offering:
- Reproducible Docker builds plus detailed vLLM and SGLang runbooks
- Benchmark tables and KLD-based quality checks
- Quantization notes and debugging of DCP/MTP/DSpark/DFlash
- PCIe topology work and a regression history
A companion Discord serves as the community workbench, making this a near-complete field manual for local deployment of very large models on PCIe-only machines.
More from Infra
- SurfSense: an open-source local NotebookLM with MCP agents and a prompt-caching lesson — FurtiveMirth · 2026-10-23
- Meta researcher: horrifying infra, not talent, is why OSS can't catch closed frontier — A_K_Nain · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23
- LatentLoop: asymmetric compute allocation cuts looped-MoE training cost by 19% — pmttyji · 2026-10-23
- Same AI request emits 8.5x more CO2 in Texas than in Sweden — jlundberg · 2026-10-23
- SK Hynix Pulls Plug on Solidigm US IPO as Chip Stocks Slide, Shares Down ~30% This Quarter — rohanpaul_ai · 2026-10-23