Running Qwen3.8 for agentic coding on 2x5090: vLLM is the only stack that doesn't trash the cache
TooManyPascals · reddit · 2026-10-21
A developer running a local 2x5090 + 128GB RAM setup shares that vLLM is the only inference framework that doesn't trash the KV cache for agentic coding with Qwen3.8-27B, currently using the INT6-Mixed-AutoRound quant with good results. He wants to try Flash Next but notes serving it on 2x5090s — where everything doesn't fit in VRAM the way 27B does — still lacks a balanced recipe, and asks the community for working configurations.
More from coding & agent
- Coding agent caught slacking admits it went off-task chasing leftover Chinese strings in ECharts — teortaxesTex · 2026-10-21
- Editing video headless with Opus + Python + FFMPEG beats any NLE, dev claims — _AustinCalvert_ · 2026-10-21
- Training Factory open-sources QLoRA fine-tuning on 4-bit CQ-quantized models — textclf · 2026-10-21
- Hidden-orchestrator multi-agent debate on free-tier APIs: opposing personas beat identical ones — Adept_Direction2591 · 2026-10-21
- T3 Connect outage: Theo says surge of users hit Cloudflare rate limits, now fixed — thsottiaux · 2026-10-21
- Anthropic reportedly built Scion to replace Git for 30,000 daily agent merges — Capatain_CV · 2026-10-21