One RTX 3090 ran Qwen 27B autonomously for 3 weeks — it shipped working CUDA kernels
skeole · reddit · 2026-09-21
Reddit user skeole ran quantized Qwen 27B on a single RTX 3090 with a local agent loop for 21 days, tasking it to build a CUDA inference engine for its own GPU. Highlights:
- Setup: Q4 quantized 27B, 200k context, deepseek-style harness, an explicit rulebook (roles, escalation, don't copy llama.cpp, don't declare the task impossible). The author doesn't write CUDA.
- Result: Working kernels and benches, but prefill 250 tps vs llama.cpp's 700 — won the process, not the perf.
- Suicide loop: Same GPU hosts both the agents (vLLM) and the engine under test; a subworker ignored the handoff script and killed vLLM twice, crashing its own brain.
- Local tax: 180 subagents, 230M tokens, 699 compactions eating 83h (17% of wall time).
Takeaway: a quantized 27B can hold a coherent engineering goal for weeks on one consumer card — protocol design matters more than model strength. 15GB dump + rulebook open-sourced on HuggingFace.
Related event: Qwen 27B Runs 21 Days on a Single RTX 3090 to Write Its Own CUDA Kernel(2 posts)→
More from coding & agent
- minpts-check: one-run predictor tells if your HDBSCAN clusters survive min_samples changes — leland_mcinnes · 2026-11-12
- Ten wild Gemini 4 Carbon demos in 24 hours: CUDA meadows, V8 engines, mech stomps — socialwithaayan · 2026-11-12
- How to cut Fable 5.2 token costs by 90%: Arc routing plus a lean CLAUDE.md — EXM7777 · 2026-11-12
- Hamel Husain's complete AI evals framework for PMs and engineers — aakashgupta · 2026-11-12
- AI coding in one year: from unusable to full automation with no code review — angelaqdai · 2026-11-12
- Interactions Tool Searches Web Animations by Motion for Claude Code to Rebuild — yuwen_lu_ · 2026-11-12