Qwen3.6 27B Fits Into 12GB VRAM
immersive-matthew · reddit · 2026-07-15
A Reddit post introduces a new Ternary version that compresses Qwen3.6 27B to a smaller size, allowing it to run locally on 12GB VRAM GPUs.
- The author states this Bonsai 27B version is updated from Qwen3.6 27B, featuring a smaller file size and significantly reduced memory usage, officially requiring only about 1/10 of the original FP16 model's memory.
- The tradeoff is slightly increased computational overhead, but it reportedly retains around 95% of the original model's benchmark performance.
- The post also mentions that the Binary version is small enough to fit into mobile devices and includes a YouTube tutorial to help users easily deploy it locally.
Related event: Bonsai 27B: The first 27B model that runs on phones(15 posts)→
More from Infra
- RTX 5080 price jumps from $1658 to $2318 in ten days on Amazon — Frizzy-MacDrizzle · 2026-10-27
- ComfyUI on 12GB VRAM: measured peaks show offloading turns 24s into 11 minutes — Impossible-Mail-4619 · 2026-10-27
- Neon rebuilds its docs search entirely on its own stack: Lakebase Search, Functions, AI Gateway — matei_zaharia · 2026-10-27
- PyTorch details 2.12x faster on-device streaming ASR on Arm SME2 via ExecuTorch — PyTorch · 2026-10-27
- NVIDIA shows speculative decoding speeding up LLM inference on Jetson with Qwen3.5 9B — NVIDIA Developer · 2026-10-27
- Revenue per GPU can rise as token prices fall, argues Gavin Baker — GavinSBaker · 2026-10-27