EXL3 Quantization Test: Running 30B Model on 12GB VRAM Smoothly
PyaesoneP · reddit · 2026-08-30
The author shares experience running Muse Glimmer 30B EXL3-SC 3.00bpw H4 fully resident on a 12GB VRAM GPU at 100K context. It achieves 30 tok/s and shows negligible quality difference compared to the official 17GB K-quant for Hermes Agent use cases. The author also tested Qwen 3.8 27B at SC2.20bpw H3, finding it usable but preferring Unsloth UDQ4KXL for coding tasks.
More from coding & agent
- Robot startup ditches VLAs for Opus-based reasoning layer in customer deployment — micoolcho · 2026-10-16
- Open-source ebook2audiobook turns any ebook into a free audiobook, 20.4k GitHub stars — thisguyknowsai · 2026-10-16
- Composing music with math patterns via Gemini agents in Antigravity CLI — fofrAI · 2026-10-16
- DuoDiscover, a Google-Berkeley-Stanford agent project, tops NVIDIA SOL-ExecBench — berkeley_ai · 2026-10-16
- Write the Agent Loop by Hand First: What LangGraph Actually Adds — m3m3o · 2026-10-16
- $20-plan users have no idea how far AI agents have come in 10 months — realmeetjames · 2026-10-16