MiMo V2.6 Flash ported to Windows sglang on a 6-GPU 5090/3090 cluster
comperr · reddit · 2026-09-27
Reddit user comperr demonstrates running MiMo V2.6 Flash via sglang on Windows, split across a cluster of 4x RTX 5090 plus a 3090 and 3090 Ti. The port required NCCL 2.32.3, sgl-kernel built under MSVC 19.44, POSIX shim patches, Gloo rendezvous, a custom a2a backend with EP via FusedMoE/Triton, and a Marlin MoE runner.
More from Infra
- Polymarket puts 29% odds on any US state enacting a data center moratorium this year — Polymarket · 2026-09-27
- Big 4 AI capex hits 2.4% of GDP, double the telecom bubble peak — but still below railroads — rickasaurus · 2026-09-27
- Open-source MLX.fast project claims up to 3x faster small-model inference on Apple Silicon — VagabondTruffle · 2026-09-27
- Goldman: token demand to grow 18x by Sept 2026, but frontier demand lags at 8-9x — rohanpaul_ai · 2026-09-27
- Banks price orbital compute at $165-180B per GW; new model says $68B by 2028 — JOBhakdi · 2026-09-27
- Modal open-sources GPU Glossary, a human-friendly dictionary for GPU programming terms — charles_irl · 2026-09-27