RTX 3090 local setup runs Qwen 27B at 100 t/s, 5x faster than MacBook M5 Pro
gutard · reddit · 2026-09-14
A dev replaced his MacBook M5 Pro 48GB with a Linux box + RTX 3090, running Qwen 3.8 27B (Q4) via vLLM at 100 t/s vs 20 t/s on Mac with llama.cpp. He serves it over an OpenAI-compatible endpoint for a custom desktop app, finds it good enough to drop his Claude subscription, and open-sourced the setup.
Related event: RTX 3090 Runs Qwen 27B Locally at 100 Tokens/s(2 posts)→
More from Infra
- Researchers encode 200+ properties of SGLang inference server, uncover 10 bugs — LingmingZhang · 2026-10-31
- Classical infrastructure gets more important, not less, in the age of AI agents — graceisford · 2026-10-31
- Lux Capital hosts closed-door dinner on AI infra, agents and security with Cognition and Modal — graceisford · 2026-10-31
- FastVideo ships install guide for NVIDIA DGX Spark, no aarch64 wheels needed — victormustar · 2026-10-31
- Intel shifts 18A capacity to Intel 3 to expand Granite Rapids as Clearwater Forest ships — zephyr_z9 · 2026-10-31
- Modal launches Sticky Sessions: serverless containers that keep state across requests — graceisford · 2026-10-31