Speculative decoding makes Qwen3.6/3.8 much faster, latest draft model adds ~10%
teortaxesTex · x · 2026-11-05
Developer hikettei shared results of making Qwen3.6/3.8 "go brrr" via their speculative decoding research, crediting a teammate's Metal optimization work, and is looking forward to Qwen4. norpadon adds that those are relatively old benchmarks — with the latest released draft model it's another 10% faster.
More from Infra
- Hugging Face shares its model optimization pipeline: profile, torch.compile, ship open kernels — ariG23498 · 2026-11-05
- Reddit proposal: use agents to digest fragmented custom inference engines into reusable building blocks — nasone32 · 2026-11-05
- LZ4 v1.11.0 ships multithreaded compression, hits 7,550 MB/s with 16 workers — jedisct1 · 2026-11-05
- Engineer praises turbopuffer as community 'sandbox tier list' makes rounds — davidcrawshaw · 2026-11-05
- Indie pretraining run hits 10T tokens of data, locks in Apple M5 and DGX Spark support — jon_durbin · 2026-11-05
- HunyuanVideo 1.5 T2V FP8 outputs noisy images on AMD RX 7900 XTX — Fragrant-Throat7048 · 2026-11-05