JetBrains ships Mellum2.1 Thinking MTP head: 1.4–1.8x faster decoding with ~90% draft acceptance
pmttyji · reddit · 2026-11-02
JetBrains released an MTP (multi-token prediction) variant of Mellum2.1-12B-A2.5B-Thinking on Hugging Face. The MTP head — one extra transformer block trained jointly with the model — drafts the next token during inference; every drafted token is verified by the main model, so output stays identical while decoding runs 1.4–1.8x faster with 90% of drafts accepted.
Weights live in model-mtp.safetensors under the mtp. prefix, announced via numnextnpredictlayers: 1 in config.json. Usage:
- vLLM 0.33+: --speculative-config '{"method": "mtp", "numspeculativetokens": 1}'
- transformers 5.20+: pass usemtp=True to generate
- llama.cpp: use the GGUF repo with --spec-type draft-mtp --spec-draft-n-max 1
More from Infra
- Fine-tuning Qwen3.5-9B on a $500 budget: is a P40 24GB enough for QLoRA? — exaknight21 · 2026-11-02
- KKR, Singtel and Tata put India data center firm STT GDC India up for sale at over $5.5B — HarveenChadha · 2026-11-02
- GPU rental rates crash as supply surges: H100 SXM falls below $2.50/hr on alternative clouds — SumitGup · 2026-11-02
- OpenAI reportedly plans to rebrand data centers as "AI compute campuses" amid public backlash — Polymarket · 2026-11-02
- AMD pours engineering and compute into vLLM to route around NVIDIA's CUDA moat — FinanceYF5 · 2026-11-02
- GMKtec Evo X5 packs 192GB of LPDDR5 on AMD Gorgon Halo for local LLM inference — Thrumpwart · 2026-11-02