AMD brings EAGLE-4 speculative decoding to Instinct MI355X day 0, 1.6x faster than built-in MTP
hongyangzh · x · 2026-10-21
AMD AI and EAGLE Corp brought EAGLE-4 speculative decoding to AMD Instinct MI355X GPUs with day-0 support. The trick: 'let the drafter measure the hardware' — EAGLE-Harness profiles drafting and verification costs at startup, then dynamically picks the optimal guess tree for the workload.
On Qwen3.8-27B (BF16, single GPU): 300 tok/s per request, 1.6x faster than built-in MTP, performance holds from 1-32 concurrent sessions, and lossless vs. plain decoding — including Gated DeltaNet layers via FlyDSL tree verification. Launch scripts for SGLang and ATOM are available.
More from Infra
- vLLM's specialized decode path hits 829 P90 TPS per user on four B300 GPUs, 26% faster — vllm_project · 2026-10-21
- vLLM says it can reproduce DeepSeek's inference performance, yielding $39/GPU/hr revenue — vllm_project · 2026-10-21
- vLLM matches DeepSeek's DeepInfer decode performance within 6%, generating $39/GPU/hr at off-peak pricing — vllm_project · 2026-10-21
- TokenGate: learned token pruning speeds up LLM prefill up to 11.3x — Liana Patel · 2026-10-21
- Modal revives multiplayer game Curvytron to demo its new Sticky Sessions routing — charles_irl · 2026-10-21
- Seroter's reading list: TorchTPU open source, Claude Code's code review system — rseroter · 2026-10-21