Disable MTP for 20-30% faster local LLM inference on limited RAM
InternalMode8159 · reddit · 2026-11-02
A user who does extensive local inference testing reminds everyone that MTP (multi-token prediction) can actually hurt performance when running very large models on constrained RAM/VRAM — and many apps now enable it by default.
In their tests on a 3060 with 32GB RAM using Gyrostrata/Strata with gyro-m and swift 1.5, simply disabling MTP yielded a 20-30% improvement in tok/s. Users with similar hardware setups are advised to try toggling it off.
More from Infra
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02
- Intel plunges 29%, worst day since the 1970s, wiping $12B off US government stake — emmanuelvivier · 2026-11-02
- Researcher hoards nearly 1TB of DRAM, refuses to sell a stick to data centers — kevinnbass · 2026-11-02