Qwen 27B now runs on 8GB Mac via Unsloth's extreme quantization
SumitGup · x · 2026-08-20
UnslothAI released a Dynamic 3.0 ultra-quantized version based on Qwen3.8-27B. Using 1-bit / 2-bit quantization (UD-IQ1M / UD-IQ2S), the model size is compressed to 6G8GB, allowing 8GB Macs to run 27B-scale multimodal LLMs while retaining multimodal capabilities and chain-of-thought reasoning.
How to run:
- Ollama: One-click run for beginners.
- llama.cpp: Terminal execution for power users via Homebrew.
Related event: Unsloth's Dynamic v3.0 Quantization Brings Qwen3.8-27B to 8GB Machines(5 posts)→
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16