Unsloth ships Qwen3.8-Flash-Next local guide: 125B MoE runs on 75GB RAM, 1-bit is 79% smaller
gnukeith · x · 2026-11-05
gnukeith discusses running Qwen3.8-Flash-Next on modest hardware: 1Q quant with n-gram offloaded to SSD and MTP on GPU. Per the linked Unsloth docs, the model is a 125B MoE multimodal release on the new Qwen4 architecture with 262K context, claimed to beat Claude-4.6-Opus (Max). It runs locally on 75GB RAM machines with no GPU VRAM—the 1-bit GGUF is 79% smaller than BF16 (355GB) while retaining 80% top-1 accuracy. New Unsloth Dynamic V3.1 quants and MTP bring 1.3-1.7x speedups via llama.cpp or the new Unsloth Desktop app; benchmarking consumed 26.2B tokens across Terminal Bench 2.1, LiveCodeBench v6, GPQA Diamond and more.
Related event: Dev shares low-end tricks for running Qwen3.8-Flash-Next and RLM concept(3 posts)→
More from Infra
- Decentralized RL Works: Untrusted Miners Lift LiveCodeBench 39.5→58.7 on Bittensor Subnet — bittingthembits · 2026-11-05
- Atomic Chat launches open-source Local AI Server tying models and agents together — SucceededMind · 2026-11-05
- Developers report Grok API instability: 13.58% error rate and 13.6s p90 latency in dashboards — ChrisUniverse · 2026-11-05
- Even cutting forecasted data center growth by 60% leaves Nvidia supply constrained — robleclerc · 2026-11-05
- Free live paper discussion: why speculative decoding is fast despite doing more compute — doesdatmaksense · 2026-11-05
- Airbnb: fine-tuned 2B model matches frontier classifiers at 1% of the cost — Ahmad_Al_Dahle · 2026-11-05