EAGLE-4 deep dive: up to 5.21× agentic coding speedup with lossless speculative decoding
zhyncs42 · x · 2026-10-21
LightSeek published a technical deep dive on training and serving EAGLE-4: drafters are trained with TorchSpec and served with TokenSpeed, which supports EAGLE-Harness dynamic trees and chains, trees sampled at T>0, and a cost-measurement toolchain. Speculation is lossless.
- Performance: on multi-turn SPEED-Bench, Qwen3.8-27B BF16 on 4×B300 (TP4) decodes at 750 tokens/s for a single request at T=1 — 2.85× over no speculation and 1.28× over the model's own MTP drafter; 3.0× on agentic coding with sampling, up to 5.21× for greedy single-user
- Architecture: keeps EAGLE-3.1's normalized single-layer drafter reading three separately normalized target hidden states plus embeddings; adds adaptive trees/chains (EAGLE-Harness decides per depth whether to go deeper based on measured cost), an efficient low-rank top-128 LM head, and SWA/MoE support
- Flow: drafter builds a token tree depth by depth; the target verifies top-n nodes in one forward pass, accepting the longest agreeing path plus one bonus token
More from Infra
- ai& claims to be Japan's biggest dedicated inference platform with 100MW+ data centers — DavidBennett__ · 2026-10-21
- SpaceX Reportedly Adding 10GW of Compute by 2027, Up from 1.7GW Today — beffjezos · 2026-10-21
- Musk's Hunt for Rentable GPU Compute Turns Into a Real-Life Cheese Shop Sketch — XFreeze · 2026-10-21
- Cloudflare discloses compromised eMudhra intermediate certificates, urges patching — Cloudflare Blog · 2026-10-21
- Shotwell: Starlink now contracted for ~12,000 planes, a quarter of available aircraft — XFreeze · 2026-10-21
- Samsung Electro-Mechanics signs $260M substrate deal with Micron as AI shortage spreads down the stack — demian_ai · 2026-10-21