Signal65: one setting change on NVIDIA GB300 NVL72 adds up to 64% output throughput, 39% lower token cost
ryanshrout · x · 2026-10-02
Signal65 published its first rack-scale NVIDIA GB300 NVL72 tuning results from PINNACLE, its agentic AI benchmark: with model, nodes, and load held fixed, software-only changes deliver large gains.
- FP8 KV cache added 59–64% output throughput on large MoE models;
- A different GPU split added up to 61% on GLM-5.3-Flash — up to 39% lower cost per output token on hardware already owned.
PINNACLE measures correct work: deterministic code-scored grading against a runtime-generated answer key (no model judges), procedurally regenerated sandboxes to prevent memorization, and live multi-step agentic sessions rather than replayed traces. NVIDIA's Ian Buck and AMD both endorsed the benchmark. Takeaway: an untuned serving stack leaves enterprise rack capacity on the table.
More from Infra
- tandem.cpp v0.3.0: 2-6x faster prefill for models too big for VRAM, Maya DeepSeek-V4.1 support — smflx · 2026-11-16
- Developers poll real costs of running coding agents on remote VMs for work — radim11 · 2026-11-16
- Datadog's new gateway returns 402 when a model's budget runs out—so agents just retry — YvesMulkers · 2026-11-16
- Max-quality local Minimax H3 video on a 4090: community-tested int8 setup — newaccount47 · 2026-11-16
- Local V2V outfit transfer on a 16GB M3 MacBook Air: 15.7 min cut to 2.4 min — might_thinker · 2026-11-16
- FlashStates hits 3.14x vLLM prefill throughput for hybrid-architecture agentic inference — 机器之心 · 2026-11-16