Running Qwen 27B locally on RTX 4090: beats pre-2025 coding models, RAM is the wall
julianharris · x · 2026-09-23
A first-hand account of running Qwen 27B locally on an RTX 4090 24GB for coding:
- Capable enough: vastly better for coding than anything before November 2025; local speeds often double Opus.
- Multi-session: switching flushes the cache and runs one at a time; Qwen 4's cache structure is messier and harder to share.
- Image processing eats huge memory.
- Slowdown: dense models halve in speed as context doubles (16-32-64-128k); Flash next / Qwen 4 is better.
- Session resumption locally can take one to two minutes.
Conclusion: memory is the biggest bottleneck, and it's a bad time to want lots of it.
More from coding & agent
- Stripe launches claimable sandboxes: frictionless payment integrations for AI agents — jeff_weinstein · 2026-11-12
- Databricks launches autonomous AI coworker Genie: 91% on enterprise tasks with half the tokens — Yuchenj_UW · 2026-11-12
- Loom founder launches Roma 2.0, a shared to-do list that orchestrates AI agents — thetripathi58 · 2026-11-12
- Developer proposes a 'blast radius' test for AI agents before building — Few-Garlic2725 · 2026-11-12
- Sosumi Eval: Apple Docs Access Lifts Coding Agent Pass Rates from ~33% to 94-98% — JordanMorgan10 · 2026-11-12
- CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench — JeremyCMorgan · 2026-11-12