CEO fixes MoE kernel index overflow bug in sonic-moe
rosstaylor90 · x · 2026-08-15
Ross Taylor, CEO of GenReasoning, shared a debugging session where he fixed a train-inference mismatch in the sonic-moe kernel.
- Context: Mismatch occurred during RL on a 120B hybrid MoE model.
- Root Cause: Index overflow in the backward db2/ds kernel where uint32 indices wrap when $T \times H \ge 2^{32}$.
- Fix: Changed tokenindices and scatterindices to int64 for the backward pass.
- Action: PR #70 submitted to Dao-AILab/sonic-moe to complete the fix previously addressed only for the forward path.
Related event: GenReasoning CEO Fixes Sonic-Moe Kernel Bug, Shares T.I.M Debugging Lessons(2 posts)→
More from Infra
- Dual R9700 setup for Qwen3.8-27B FP8 inference via vLLM — kapteinpyn · 2026-08-15
- User seeks to buy used A100 GPUs — beffjezos · 2026-08-15
- AI hits physical limits: bottlenecks shift to land, power, and time — r0ck3t23 · 2026-08-15
- Oracle's Stargate AI Data Center Gas Pipeline Delayed to Feb 2027 — Polymarket · 2026-08-15
- Core AI Model Zoo Enables Local LLM/VLM Inference on Apple Devices — amos_gyamfi · 2026-08-15
- NVIDIA CEO Jensen Huang announces AI factory infrastructure as new asset class with BlackRock, Blackstone, and others — ___Mufasaa · 2026-08-15