NVIDIA releases LiLiCorr speculative decoding draft for Nemotron 3.5 Bolt to speed up 74B serving
_akhaliq · x · 2026-10-17
NVIDIA released a LiLiCorr speculative decoding draft model for Nemotron 3.5 Bolt on Hugging Face. It pairs with the 74B target model — the draft accepts longer token blocks to serve faster, while the target verifies every token, improving throughput without sacrificing output quality.
More from Infra
- Modal explains ImageFS, the lazy-loaded filesystem behind sub-second cold starts — josh_wills · 2026-10-17
- Blackwell Ultra GPU rental prices climb 17 straight days to an all-time high — McDonaghMatthew · 2026-10-17
- Stop posting tok/s screenshots: 5 metrics a credible local AI benchmark needs — EAccelerate_42 · 2026-10-17
- 3 models, 4 engines, 4 Macs: Draw Things hits 1.8x mflux speed on Apple Silicon — liuliu · 2026-10-17
- SurrealDB's new engines hit 28.3x RocksDB point reads, Agent Memory goes live — mattturck · 2026-10-17
- Zuckerberg & Wang on Muse Wonder: Meta's $100B AI Bet Past 5GW — RylanSchaeffer · 2026-10-17