12GB VRAM Test: Qwen3.8 Q2 Usable, but MoE Remains Superior
CoffeeToCode99 · reddit · 2026-08-17
Author compared Qwen3.8-27B (Dense, Q2/Q3) vs. Qwen3.6-35B-A3B (MoE, Q4) on an RTX 5070 Ti Laptop (12GB VRAM), concluding:
Setup
- Hardware: RTX 5070 Ti Laptop, 12GB VRAM.
- Backend: llama.cpp CUDA.
- Settings: 4k context, q8 KV, --fit on, no MTP.
Key Findings
- Qwen3.8 Q2 (Dense)
- Faster speeds (Prompt 412 t/s, Gen 35.9 t/s).
- Quality dropped in sanity tests (e.g., failed bat-and-ball), proving Q2 has noticeable degradation.
- Qwen3.8 Q3 (Dense)
- Maintained accuracy (6/6) in sanity tests.
- Extremely slow generation (7.5 t/s), making it painful for interactive use.
- Qwen3.6 35B-A3B (MoE)
- Perfect accuracy (6/6).
- Fastest generation (59 t/s), significantly outpacing Dense Q3.
Conclusion
- If the goal is "can it run," Qwen3.8 Q2 works.
- For actual daily chat/coding, the 35B-A3B MoE remains the best choice on 12GB VRAM, balancing speed and quality.
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16