ExLlamaV3 Is Underrated: Better Quants, Lower KLD, Faster Than llama.cpp
Embarrassed_Soup_279 · reddit · 2026-09-08
A Reddit user argues the local inference backend ExLlamaV3 (ExL3) is underrated: in personal tests it delivers higher-quality quants, lower KLD metrics, and faster speeds than llama.cpp (NVIDIA GPUs only). CPU MoE offload was only recently added, which may explain low adoption.
- Daily driver setup: tabbyAPI exl3 backend + qwen3 27B SC 6bpw H6 and qwen3 flash-next 4bpw
- The author stresses these are personal test sets, not formal benchmarks
- Encourages more people to try the project
More from Infra
- Cloud release velocity scores: AWS 95, Azure 90, Google Cloud 82 — DavidLinthicum · 2026-10-27
- Selling a 5090 to scale local inference: one business weighs 5 hardware paths — Anarchaotic · 2026-10-27
- AWS tutorial: fleet-wide edge CV model lifecycle with SageMaker SDK v3, ~$0.07 per run — AWS ML Blog · 2026-10-27
- Qwen3.8-27B hits 40 tok/s on Jetson Thor with INT4 + MTP, accuracy beats FP16 — No-Dragonfly6246 · 2026-10-27
- Celesto launches CI runners nearly 90% cheaper than GitHub Actions, with agent sandboxes — aniketmaurya · 2026-10-27
- Supabase founders on rebuilding the backend for a billion AI-built apps — Latent Space · 2026-10-27