Running Qwen3.8-27B on 2x3090: 200K Context with F16 KV, Vision, and Reasoning
Sisuuu · reddit · 2026-08-15
A user successfully deployed the Q8KXL quantization of Qwen3.8-27B on a dual RTX 3090 setup, achieving a 200K context window with F16 KV cache and vision support enabled. The post details the memory optimization benefits of the model's hybrid architecture (16 full attention layers + 48 linear attention layers) and the specific llama.cpp configuration required to avoid OOM errors. It provides the full command line recipe for enabling speculative decoding and reasoning features, reporting final speeds of 73 tok/s for code and 58 tok/s for prose.
More from Infra
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16
- Cherry pi on a Raspberry Pi: the tiny-model rumor developers are watching — sull · 2026-10-16
- CoreWeave's debt costs falling: DDTL spreads tightened from S+9.62% to S+2.25% — BenBajarin · 2026-10-16
- SoftBank lines up US IPOs for SB Energy and AI robotics spinoff Roze, valuations top $50B — nordicinst · 2026-10-16
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16
- Hugging Face scales Indic pretraining with 2.8T tokens and an open data pipeline — Hugging Face Blog · 2026-10-16