Building a production code-repair agent: which LLM survives 60-turn, $50 fix sessions?
SmallDepartment8463 · reddit · 2026-09-07
A developer describes an autonomous code-repair agent in production: a Python monorepo extracting financial data from messy PDFs (OCR, bank statements, tax returns). When a deterministic check fails, an agent gets failure evidence, a sandboxed repo, and shell/file tools, then investigates for up to 60 tool turns, forms a hypothesis, and edits code — shipping only if it passes a deterministic judge (failing job now passes, pinned regression corpus holds to the cent, diffs capped at 40 deleted lines).
- 2h per run, $30-50 budget, latency irrelevant
- Honest "cannot fix" beats hallucinated patches; bad fixes escalate to humans
- Currently on Anthropic's top models with decent success, but hard cases still escalate
- They're asking who has run long agentic sessions (50+ turns, 100K+ token contexts): which model holds up without confidently patching the wrong thing, and which knows when to quit?
- They can swap models via config on real failed jobs and will run a bake-off on whatever the thread converges on.
More from coding & agent
- Claude Code team reportedly ditched GUI/TUI, now using claude tag for 70%+ of work — himanshustwts · 2026-09-07
- Microsoft open-sources tgrep, a trigram-indexed grep up to 52x faster than ripgrep — jedisct1 · 2026-09-07
- How should billing work when an AI system auto-selects the model? — Colddew-YJ · 2026-09-07
- SmolVM: open-source microVM sandbox runs OpenClaw 2.0 in isolation, boots in milliseconds — aniketmaurya · 2026-09-07
- Researchers formalize the AI agent attack surface: models + data + tools + permissions — JayAlammar · 2026-09-07
- Developer vibe-codes an interactive Odyssey narrative scroller with GPT-6 Astra — Pristine_Good7326 · 2026-09-07