Qwen 27B IQ3_XXS beats Bonsai ternary PQ2 by 3x on 16GB VRAM local test
Danmoreng · reddit · 2026-09-19
The author compared two locally runnable options on 16GB VRAM: Qwen3.8 27B IQ3XXS (10.18GiB) vs Bonsai ternary PQ2 (6.42GiB), using the same UI-generation tasks on identical hardware.
Quality was close (both 4/4 tasks, 20/20 assertions), but efficiency differed sharply:
- Wall time 8:00 vs 24:09 — Qwen 3.02x faster
- Output tokens 27,197 vs 84,176 — 3.1x fewer (Bonsai is chattier, hence slower)
- Decode 83.59 vs 64.91 tok/s; speculative acceptance 65.22% (MTP) vs 39.76% (modified n-gram)
Caveats: unscientific test, and llama.cpp ran with MTP while Bonsai lacks it, widening the gap. Results and repo are public.
More from Infra
- Malaysia poised to become a compute superpower with both US and Chinese chips by next year — basedjensen · 2026-11-04
- ByteDance details USDB storage system: 85.8% lower write latency for recommender sequences — _reachsumit · 2026-11-04
- Oracle cancels $7B chip leasing deal with Tencent, FT reports — SumitGup · 2026-11-04
- Inference engine shrunk to 1023 bytes, still runs SmolLM2-135M — cneuralnetwork · 2026-11-04
- Musk says space data centers are 'looking pretty appealing right now' — elonmusk · 2026-11-04
- Data centers are turning to old jet engines as the wait for power grows — IndependentCoast3974 · 2026-11-04