cuda.fast leaderboard: speculative decoding makes Bonsai 2 9B run 240.9% faster on DGX Spark
HankYeomans · x · 2026-10-17
cuda.fast is back online and published a new leaderboard entry: Bonsai 2 9B runs 240.9% faster on DGX Spark via speculative decoding.
- Scoring: single-stream composite = prefill gain^0.25 × decode gain^0.75, so decode carries 75% of the weight; serial baseline is 1.0, computed and sealed by benchd.
- Top solver ArthurOstapenko (using Opus 5.5 as drafter) hit decode 386.5 tok/s, prefill 4052.5 tok/s, 9.9 tok/round.
- Runner-ups: newjordan with GPT-6.1 Sol as drafter at 150.6%, davidtai at 117.3%.
- 11 promoted submissions from 3 solvers so far; the mlx.fast comparison is still pending.
More from Infra
- Nebius is building the next AWS; CoreWeave is a GPU datacenter OEM — kevinsxu · 2026-10-17
- If cyberspace were Earth, the biggest AI datacenter would exceed the EU — LeviTurk · 2026-10-17
- Ranking seven AI compute plays on survivability: Nebius first, CoreWeave fifth — pdamodaran · 2026-10-17
- Qwen 27B at Full 262k Context on a 16GB RX 9070 XT: 80+ t/s Decode With Adaptive KV Streaming — bottlestreet3d · 2026-10-17
- Cloudflare open-sources Pipecat voice agent on Python Workers with WebRTC — ritakozlov · 2026-10-17
- Durable PTY: Open-Source Tool Keeps AI Coding Agents Alive Across Crashes and Reboots — jasonkneen · 2026-10-17