Qwen3.8 Flash Next protip: use mmap mode for lazy loading
Pyrolistical · reddit · 2026-08-30
Sharing experience running Qwen3.8-Flash-Next in a memory-constrained environment. The default load-mode auto might not use mmap, leading to OOM. After manually enabling tensor-read-lazy on and load-mode mmap, the author successfully ran the Q4 quantized model on AMD GPUs and provided specific llama-bench commands and performance data (279 t/s).
More from Infra
- Autonomous sells personal AI computers from $26,100 that keep data in the room — dee_hw · 2026-10-15
- vLLM's bitwise-consistent RL: zero KL on Nemotron NVFP4, 91.3% throughput on DeepSeek-V4-Flash — DynamicWebPaige · 2026-10-15
- vLLM achieves bitwise-consistent RL training: DeepSeek-V4-Flash hits 0 diff over 100 steps — vllm_project · 2026-10-15
- vLLM Achieves Bitwise-Consistent RL Training, Fixing KL Blowups and Score Collapse — vllm_project · 2026-10-15
- Analyst Ben Bajarin: The AI Industry Needs a Lot More Compute — BenBajarin · 2026-10-15
- Next-gen GraalVM benchmarks: 4x faster startup, 2x less memory, no build step needed — RealGeneKim · 2026-10-15