Qwen 27B quant runs on 8GB VRAM: 200k context, 92.5% Top-1 vs BF16
Brief-Tap-6616 · reddit · 2026-11-21
A Reddit user released a 2.1bpw quant of Qwen3.8 27B (AmperInteger+/Trellis format) that actually works on 8GB GPUs, supporting 200k context with 92.5% Top-1 match against BF16 and coding scores within 10% of full precision — 17% above the 9B quant.
- Built on Latent Sparse Attention (pushes context cost off VRAM) and SJKVaRN (protects KV quality at 4 bits)
- Not post-trained for coding (<10M-token heal), yet tied with coding-specific Mellum 2.1 on LiveCodeBench
- A major step up over the previous IQ1M fallback, with 59–73% better KLD
- Caveats: run headless to free 500+MB, error explodes below 3bpw; supports the new HF Pi code adapter for further gains
- Author calls it a dry run for Qwen4
More from Infra
- SEC launches investigation into Nvidia's accounting practices — Polymarket · 2026-11-21
- Nvidia slides to $131, on track for longest weekly losing streak since 2008 — Polymarket · 2026-11-21
- llama.cpp PR Delivers Up to 3x Prompt Processing Speedup for Gemma 4 Multi-Slot Queries — jacek2023 · 2026-11-21
- Anthropic goes public at $83/share: $130B cash, $431B compute commitments, $77B supplier financing — LexSokolin · 2026-11-21
- From OSS to startups: how vLLM, SGLang and FlashAttention teams monetize inference research — AccBalanced · 2026-11-21
- SpaceX's GPU financing grows to $44B, largest ever, to fund NVIDIA Vera Rubin chips — XFreeze · 2026-11-21