Unsloth GGUFs run Qwen-Image-2.1 locally on as little as 6GB VRAM
danielhanchen · x · 2026-09-23
Unsloth released Dynamic GGUF quantizations of Qwen-Image-2.1, claiming the 7B text-to-image model matches Nano Banana 2.0 and runs locally on 12GB VRAM — or 6GB with FP8 via RAM offloading.
- INT8/FP8 with pinned offloading fits in 6-8GB VRAM while staying reasonably fast
- The GGUF contains only the denoiser; pair with the bf16 VAE and Qwen3-VL-8B text encoder. Measured: UD-Q4KXL encoder vs Q4KM gives LPIPS 0.029, SSIM 0.959, 5.15GB vs 5.03GB, 36.5s vs 39.0s per image
- Runs via Unsloth Desktop or stable-diffusion.cpp; model card includes a full sd-cli command (1024x1024, 20 steps, cfg 6.0, euler)
- Qwen-Image-2.1 itself is a unified text-to-image and editing model with 32 Single-Stream DiT layers
Related event: Unsloth's quantized Qwen-Image-2.1 runs locally on just 6GB VRAM(2 posts)→
More from Infra
- Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
- a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
- AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
- Tsinghua's BaGuaLu-2 tames MoE expert-load spikes at runtime, accepted to SC26 — jiqizhixin · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12