SemiAnalysis token-throughput-per-watt chart sparks debate over MI355X MTP accuracy issues
nrehiew_ · x · 2026-10-12
A SemiAnalysis chart comparing token throughput per all-in utility MW vs interactivity for DeepSeek R1 0528 inference sparked discussion on X. It benchmarks GB300 NVL72, B300, B200 (Dynamo TRTLLM with MTP), MI355X (MoRI SGLang), and Jalapeño (Teacup), using per-chip power figures from 1.37kW (H100) to 3.3kW (VR200).
nrehiew questioned why the MI355X MTP datapoint is flagged with accuracy issues (open SGLang issue since June 2026), and whether inference providers use MTP internally. The chart also notes disaggregated configs count power per decode/prefill chip, making direct comparisons imperfect.
Related event: Debate swirls over Jalapeño energy-efficiency claims and MTP usage(4 posts)→
More from Infra
- Same model, different behavior: Reddit asks how to test open-model inference compatibility — Top_Speaker_7785 · 2026-10-12
- The memory price crisis is far worse than you think, video explains — Brassens71 · 2026-10-12
- Qwen's VisionWeave saves 43% visual tokens at 98.9% performance, 2.3x throughput — Qwen · 2026-10-12
- Dev Picks Up AMD Halo Mini PC, Praises Its Heat Management for Local AI — gajesh · 2026-10-12
- SoloQ: calibration-free 4-bit quantization cuts dLLM memory 2.61x, speeds inference 2.24x — UniversityofSouthernCalifornia · 2026-10-12
- HF engineer shows off 168.5TB home storage server built with RAID10/RAID6 mix — TheZachMueller · 2026-10-12