Qwen3.8 IQ1_S on RTX 5070: Achieving 22t/s with 12GB VRAM
jacek2023 · reddit · 2026-08-29
User reports running Qwen3.8-Flash-Next with extreme IQ1S quantization on a single RTX 5070 (12GB VRAM).
Setup & Command
- Model: Qwen3.8-Flash-Next-UD-IQ1S (GGUF)
- Hardware: Single RTX 5070
- Args: --parallel 1, -c 10000
Performance Metrics
- Generation speed: 21-22 tokens/s
- Prompt processing: 25 tokens/s
- VRAM: Fits within 12GB
The user recommends starting with small quantizations for low-VRAM setups to verify compatibility.
More from Infra
- Autonomous sells personal AI computers from $26,100 that keep data in the room — dee_hw · 2026-10-15
- vLLM's bitwise-consistent RL: zero KL on Nemotron NVFP4, 91.3% throughput on DeepSeek-V4-Flash — DynamicWebPaige · 2026-10-15
- vLLM achieves bitwise-consistent RL training: DeepSeek-V4-Flash hits 0 diff over 100 steps — vllm_project · 2026-10-15
- vLLM Achieves Bitwise-Consistent RL Training, Fixing KL Blowups and Score Collapse — vllm_project · 2026-10-15
- Analyst Ben Bajarin: The AI Industry Needs a Lot More Compute — BenBajarin · 2026-10-15
- Next-gen GraalVM benchmarks: 4x faster startup, 2x less memory, no build step needed — RealGeneKim · 2026-10-15