MTP slower than standard decode? RTX 4060 user sees half throughput in llama.cpp
Aggravating-Push-207 · reddit · 2026-10-12
A Reddit user running Ornith 1.5 9B GGUF in latest llama.cpp with draft-MTP gets 15 tok/s TG and 200 tok/s PP — half the speed of plain decoding, despite a 0.95 draft acceptance rate and 2.27 mean accepted length. GPU utilization and memory bandwidth also drop by half on an 8GB RTX 4060. Full command included; the thread probes whether MTP scheduling or implementation is to blame on consumer GPUs.
More from Infra
- OpenAI chip head Richard Ho on Jalapeno: why Broadcom beat Nvidia and AMD as partner — richardjnieva · 2026-10-12
- SpaceX building $250M GigaBay at Starbase to build up to 1,000 Starships a year — DimaZeniuk · 2026-10-12
- Bittensor's TPN Labs ships first model TPN005, cutting base size ~40% with 32k context intact — const_reborn · 2026-10-12
- Lightning AI launches BYOC: run Studios on your own AWS, GCP, Azure or Slurm infra — LightningAI · 2026-10-12
- Microsoft to scale in-house Maia AI chips to multi-gigawatt inference by 2027 — The Decoder · 2026-10-12
- Microsoft plans huge deployment of its own AI chips to cut costs — pstAsiatech · 2026-10-12