RTX 5090 local test: Qwen 3.8 Flash NVFP4 hits 9K prefill after ditching Windows
Rozzemak · reddit · 2026-11-02
Reddit user Rozzemak shared a detailed local run of Qwen 3.8 Flash Next NVFP4 (Basalt format) on an RTX 5090 (32GB VRAM) with 64GB RAM:
- Switching from Windows 11 to CachyOS was the key move: Win11's 1.5GB VRAM and 6GB RAM overhead caused expert spilling to RAM and a 70% drop in decode/prefill. On Linux the box now reaches 9K tps prefill, 200 tps decode, with 256K context.
- Quality: QFN 3.8 beats dense 27B on many tasks in his experience; prefill is 2-3x faster vs ninfer and dense 27B.
- Workflow details: He fits a vision adapter alongside the model, runs a small OCR/vision model on another machine as a sidecar, and uses the oh-my-openagent harness locally.
- He sees 27B dense → 125B-A6 MoE as the best quality/speed jump, is excited for the Qwen 4 family, and asks for benchmarks from 2×5060/5070 setups.
More from coding & agent
- Claude autonomously runs 12-hour app migration, initial design would cost $3,000/month — tristanbob · 2026-11-02
- Behind the Craft lineup: OpenAI product VP, Claude designer, 10-person $32M AI company — petergyang · 2026-11-02
- Redditor adapts Google's Harness² into an offline self-improvement loop for Claude/Codex/Gemini agents — cobresalino · 2026-11-02
- Dev builds Claude-powered pipeline that auto-generates quantum computing explainer videos — AIandDesign · 2026-11-02
- Patched Strata cuts DeepSeek V4.1 agent session startup 20× on a gaming GPU — HearMeOut-13 · 2026-11-02
- Anthropic design lead says agentic AI freed him from his laptop for most daily work — davidhoang · 2026-11-02