ExLlamaV3 Is Underrated: Better Quants, Lower KLD, Faster Than llama.cpp
Embarrassed_Soup_279 · reddit · 2026-09-08
A Reddit user argues the local inference backend ExLlamaV3 (ExL3) is underrated: in personal tests it delivers higher-quality quants, lower KLD metrics, and faster speeds than llama.cpp (NVIDIA GPUs only). CPU MoE offload was only recently added, which may explain low adoption.
- Daily driver setup: tabbyAPI exl3 backend + qwen3 27B SC 6bpw H6 and qwen3 flash-next 4bpw
- The author stresses these are personal test sets, not formal benchmarks
- Encourages more people to try the project
More from Infra
- Anthropic S-1 filing reveals 85% H1 gross margin and 5,300 employees — Jsevillamol · 2026-10-26
- HBM demand shifts: low-stack inference plans shelved, Rubin to go 12-high only — BenBajarin · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26
- Ubicloud publishes "Fixing Tool-Call Accuracy in vLLM" — petrusenko_max · 2026-10-26
- Meta approves nonvoting Class C shares as 2026 capex guidance hits $135B–$145B — thione · 2026-10-26
- Unsloth Ships MiniMax-M3.1-Flash Quants: 2-bit Retains 91.5% Accuracy, Runs in 150GB RAM — danielhanchen · 2026-10-26