Single 3090 + 32GB RAM: squeezing max fidelity out of local Qwen3.8-27B
Certain_Yam_5824 · reddit · 2026-09-12
The OP details a local setup with a single 3090 (24GB VRAM) and 32GB DDR4, running Qwen3.8-27B at q80 with a 128k context window. To fit context in VRAM they had to strip the mmproj (multimodal) component from the model. They want a bigger overnight model but can't find multimodal options that fit 48GB after context overhead, currently rerunning Qwen3.8 at 256k when needed. The post includes concrete VRAM tradeoffs and asks for 48GB model recommendations.
More from Infra
- SemiAnalysis: OpenAI reportedly negotiating ~1GW TPU commitment while seeking sites — dylan522p · 2026-10-28
- I tested 11 video upscaling methods for AI video: local, Topaz and cloud compared — lopas99 · 2026-10-28
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- SK Hynix says memory demand will outpace supply through 2030 — Chemical_Hawk_6307 · 2026-10-28