llama.cpp Expert-Pool Fork Boosts Qwen MoE to 17 t/s on a Single MI50 16GB
Atretador · reddit · 2026-09-18
A llama.cpp fork (moe-expert-pool) fixes the admission-budget issue that silently disables expert offloading on 16GB cards, letting a Qwen MoE run on a single MI50. Benchmarks: stock CPU MoE path 11.76 t/s vs 16.39 t/s with 40 expert-cache slots (61.8% hit rate), 17.60 t/s warm, peak 19.8 t/s observed. Full run script with all parameters shared.
More from Infra
- Fire at Taiwan Panasonic PCB materials plant may tighten CCL supply further — zephyr_z9 · 2026-11-04
- Same Claude model served by Bedrock one day and Anthropic the next, dev finds — georgos_pappas · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04
- Gigabyte's AI TOP motherboard packs 2TB RAM and 9 GPUs for home AI labs — vincent_koc · 2026-11-04
- Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems? — infectiousstupidity · 2026-11-04
- imec neural probe integrates photonics directly into 130nm CMOS backend — jwt0625 · 2026-11-04