Dev claims ~16% extra speedup for Qwen models with MTP, open llama.cpp/mlx release planned
karminski3 · x · 2026-10-20
karminski3 says a week-long inference optimization effort could push Qwen3.8-27B, Qwen3.6-35B-A3B and Qwen3.8-Flash-Next another 16% faster with MTP enabled. He plans to ship ready-to-use llama.cpp and mlx builds if it works.
More from Infra
- EC2-Bench: Systematic Analysis of Edge-Cloud Collaboration for Long-Horizon LLM Agents — NortheasternUniversity1923 · 2026-10-20
- Everyone's fighting over models, but energy owns the AI stack — r0ck3t23 · 2026-10-20
- Renting GPUs Is No Longer Special: Neoclouds Buy Software to Build Moats — FinanceYF5 · 2026-10-20
- Neoclouds' New Playbook: GPU Rental Is Commoditized, So They're Buying Software Moats — FinanceYF5 · 2026-10-20
- Jensen Huang: Nothing pains me more than Germany falling so far behind in technology — 2C_ornot2C · 2026-10-20
- Budget dual-GPU over PCIe x1 to run a 31B dense model: seeking advice — Introvertosaurus · 2026-10-20