Six load forecasters benchmarked on GPU-hours: none beat the last-value baseline
Vegetable-Top-3670 · reddit · 2026-09-03
The author built a harness scoring load predictors for NVIDIA Dynamo's SLA Planner on GPU-hours and SLO violations (not MAE), sweeping each predictor to the cheapest headroom that hits the same 1.0% violation target before comparing cost.
On 400 intervals of a BurstGPT trace: oracle (perfect foresight) costs 73.85 GPU-hours; the do-nothing last-value baseline hits 175.00 at 0.49% violations; TimesFM 3.0 ties at 176.85; Chronos is 50% worse than doing nothing at 264.05; ARIMA (Dynamo's shipped default) costs 501.95; Prophet never hits the target at any headroom (1253.95).
Key takeaways: perfect one-step foresight is 50-58% cheaper than last-value—real money on the table—but six forecasters including two time-series foundation models captured none of it. The author's read: for one-step-ahead provisioning, the last observation already contains most of the available signal, and the single-number interface caps upside before model choice matters. Switching to P90 quantile provisioning didn't help either. Honest caveats: single window, analytic simulator with uncalibrated engine profile (only relative comparisons valid), coarse headroom grid. Repo is MIT with 200 tests; an issue was filed upstream on Dynamo's ARIMA default.
More from Infra
- Loudoun County's 20-year data center history previews America's AI infrastructure future — suchenzang · 2026-09-03
- Alexandr Wang mocks NIMBYs: 'Planes come and go, but the data center hum never stops' — suchenzang · 2026-09-03
- GLM-5.3-Flash (320B MoE) on two DGX Sparks: rewritten CUDA kernel boosts fat-expert GEMM by ~40% — EAccelerate_42 · 2026-09-03
- Beff Jezos: Biology's Compute-per-Watt Is Massively Underestimated, Bio-Silicon Complexification Begins — beffjezos · 2026-09-03
- AMD's official Qwen3.8 MXFP4 quant fails to load in llama.cpp, is support still missing? — mailto_devnull · 2026-09-03
- Should LLM tokens carry green data-center validation labels, like Fair Trade? — jdavid · 2026-09-03