Qwen 27B Compressed Down to 3.9GB
max_paperclips · x · 2026-07-17
The reposted content discusses PrismML's Bonsai 27B: an aggressively quantized version of Qwen3.6 27B that can be loaded onto a single 3090, with claims that it can even run on mobile devices.
Core details include:
- No retraining required; it simply applies binary/ternary quantization to existing weights.
- The 1-bit version reportedly compresses the 27B model down to roughly 3.9GB, while a ternary version sits at about 5.9GB.
- The author stresses that this compression keeps the model—which typically "collapses below 4bit"—fully usable, claiming it retains around 89.5% of its original intelligence.
- The reposter adopts a trust-but-verify approach, planning to test the 1-bit version against the original Q4 version using the same agent tasks and benchmarks to see exactly where degradation occurs.
Related event: Quantized Qwen 27B Model Runs on iPhone(2 posts)→
More from Infra
- Alchemy adds effect/Workflow support across Durable Objects, Workflows and Lambda Durable Functions — samgoodwin89 · 2026-10-27
- Alchemy's Lambda Web Functions run plain Node web servers on Lambda, no OpenNext needed — samgoodwin89 · 2026-10-27
- Foldkit Adds SSR Support With a Few Lines of Config on Cloudflare and AWS — samgoodwin89 · 2026-10-27
- Alchemy Cuts Cloudflare Bindings CPU Time by Up to 166× Via Profiling — samgoodwin89 · 2026-10-27
- Importing all 438 AWS services now takes 13.9 MB and 0.14s, down from 227 MB and 1.61s — samgoodwin89 · 2026-10-27
- Alchemy makes all ~2,850 AWS bindings work from any cloud, not just S3/Lambda/DynamoDB — samgoodwin89 · 2026-10-27