Qwen3.8-Flash-Next runs stable at 1.66bpw exl3, fits entirely on one 5090

Unstable_Llama · reddit · 2026-10-21

Reddit user UnstableLlama released a 1.66bpw exl3 quantization of Qwen3.8-Flash-Next (open on Hugging Face) and stress-tested its stability.

• The model fits entirely on a single 5090 within 32GB VRAM, with no offloaded experts — a rare example of stable sub-2bpw quantization.
• Inference was stable throughout with no reasoning loops at 200-300 tokens/s; even with KV cache quantization, a 32GB GPU leaves tight context room requiring frequent compacting.
• The author included an interactive "fluid cat" demo video generated by the quantized model.

Original post →

More from Infra

• DeepSeek docs reveal cross-datacenter inference tradeoffs: fiber 'easily severed by construction' — teortaxesTex · 2026-10-21
• NVIDIA GTC 2026 kicks off in Berlin with supply and volatility on stage — kimmonismus · 2026-10-21
• Data center power set to double by 2030 as grid queues top 2,500 GW — ingliguori · 2026-10-21
• Interactive chart plots Artificial Analysis Intelligence Index against model parameter count — xrailgun · 2026-10-21
• antirez tries DHH's Omarchy on a Dell XPS, plans to adapt DwarfStar SSD model streaming — antirez · 2026-10-21
• NVIDIA reportedly sits on $35B in China orders but can supply only 5-10% — emmanuelvivier · 2026-10-21

Infra channel →