Qwen 3.8 Flash offloads ngram table to SSD via SGLang with zero loss
Easy_Werewolf7903 · reddit · 2026-08-29
A Reddit post highlights a technique where the Qwen 3.8 Flash model offloads its Next ngram lookup table to SSD, streamed via SGLang. The user notes it looks promising and surprisingly incurs no performance loss, inviting others to verify the results.
More from Infra
- Autonomous sells personal AI computers from $26,100 that keep data in the room — dee_hw · 2026-10-15
- vLLM's bitwise-consistent RL: zero KL on Nemotron NVFP4, 91.3% throughput on DeepSeek-V4-Flash — DynamicWebPaige · 2026-10-15
- vLLM achieves bitwise-consistent RL training: DeepSeek-V4-Flash hits 0 diff over 100 steps — vllm_project · 2026-10-15
- vLLM Achieves Bitwise-Consistent RL Training, Fixing KL Blowups and Score Collapse — vllm_project · 2026-10-15
- Analyst Ben Bajarin: The AI Industry Needs a Lot More Compute — BenBajarin · 2026-10-15
- Next-gen GraalVM benchmarks: 4x faster startup, 2x less memory, no build step needed — RealGeneKim · 2026-10-15