Indie dev ternary-quantizes Qwen 27B to 7.3GB, keeps image skills for ComfyUI
udmrzn · x · 2026-10-01
- A Japan-based developer (a practicing physician) released Mitsuba, a ternary-quantized, ComfyUI-specialized version of Qwen3.8-27B: 51.5GB shrunk to 7.3GB (14% of original), reportedly the first personal ternary quantization release of its kind in Japan.
- It does one job: look at an image, then write prompts for image/video generation (e.g. with Krea 2 I2T2I). Image recognition drops only 89.8→87.8; fully-compliant generation prompts improve 3/10→6/10; coding collapses 66→4.
- Requires PrismML llama.cpp, Thinking recommended OFF; uncensored version in progress.
- The thread argues purpose-built quantized models may soon let 27B-class VLMs run on 8GB VRAM.
More from Infra
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72 — bookwormengr · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16