Lessons from running a local Qwen 27B overnight on long-horizon coding across 2 GPUs
paulqq · reddit · 2026-09-26
A developer ran a local Qwen 27B on llama.cpp (tensor-split across an RTX 4080 SUPER + RTX A4000) inside a deepseek harness for about a day, building two small agent-harness tools, and shared detailed lessons.
What worked
- Memory as files, not context: a status file rewritten every step served as the resume interface; context compaction mid-build didn't matter.
- Small slices, verified: machine checks first (tests, typecheck, curl), human review second, never more than one slice ahead of last verification.
- A throwaway port for testing; the main UI and LLM server were never touched.
- Real UI bugs came from human screenshots, not the model's "vision."
What didn't
- No browser: "visual verification" meant grepping a minified JS bundle.
- The model shares the inference slot it monitors — a careless POST makes it queue behind itself; fixtures only.
- Loading another fat model on the same GPUs risks OOM; read-only probes only.
- A mid-build Q6→Q4 quant swap staled every number in its own notes.
- Tests written against a nonexistent contract produced a pile of red tests; stop and write the split down.
- Stale host + new client = blank pane; null-guards plus a regression feeding old JSON shape.
He frames it as pairing: the model wrote most code, the human set rules, accepted/rejected, and made the ship call.
More from coding & agent
- LangChain on Building Prod Systems with Jev and LangGraph — LangChain · 2026-09-26
- Dev uses open-source Ling-3.0-flash-VL to let AI redesign the foldable iPhone in a single HTML file — alifcoder · 2026-09-26
- TypeSafe's decision model Jev claims 200x speed, 400x cost savings on agent routing — adnan_hashmi · 2026-09-26
- Anthropic launches Claude plugin directory portal as MCP usage jumps 110x this year — ClaudeDevs · 2026-09-26
- Open-source Jev agent plays Pokemon Red live, pushing fast-decision AI beyond Tetris — supportingthedogs · 2026-09-26
- Anthropic deep dive: effort tuning in Claude Code pays off most for security and code review — trq212 · 2026-09-26