Squeezing Qwen 3.8 27B MTP Q8_0 + Vision into 48GB VRAM

Creative-Type9411 · reddit · 2026-08-17

A user shared specific parameters and configurations for running the Qwen 3.8 27B MTP Q80 + Vision model within 48GB VRAM (3x Tesla T4). By fine-tuning settings (e.g., disabling mmap, using mlock, draft-mtp), they achieved 19.2k context at 35t/s without offloading, asking others for optimization tips.

Original post →

More from Infra

• First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
• Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
• NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
• 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
• Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
• Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16

Infra channel →