Building a production code-repair agent: which LLM survives 60-turn, $50 fix sessions?
SmallDepartment8463 · reddit · 2026-09-07
A developer describes an autonomous code-repair agent in production: a Python monorepo extracting financial data from messy PDFs (OCR, bank statements, tax returns). When a deterministic check fails, an agent gets failure evidence, a sandboxed repo, and shell/file tools, then investigates for up to 60 tool turns, forms a hypothesis, and edits code — shipping only if it passes a deterministic judge (failing job now passes, pinned regression corpus holds to the cent, diffs capped at 40 deleted lines).
- 2h per run, $30-50 budget, latency irrelevant
- Honest "cannot fix" beats hallucinated patches; bad fixes escalate to humans
- Currently on Anthropic's top models with decent success, but hard cases still escalate
- They're asking who has run long agentic sessions (50+ turns, 100K+ token contexts): which model holds up without confidently patching the wrong thing, and which knows when to quit?
- They can swap models via config on real failed jobs and will run a bake-off on whatever the thread converges on.
More from coding & agent
- Dev demos AI models handling 3D objects and animations surprisingly well — chongdashu · 2026-10-23
- Jason Kneen burns 104.56B tokens and $24.7K in 304 days of AI coding on Codex and Claude Code — jasonkneen · 2026-10-23
- Typosquatted Blender MCP Repo Hit 958 Stars in Days and Silently Ran Malware — Promptmethus · 2026-10-23
- Benchmarking 56 fresh dependency bugs: MCP routing lifts Claude Code accuracy 68% to 91% — Available_Side_7134 · 2026-10-23
- Swapping only the eval harness moves results 30%: designing agent evals like a scientist — AI Engineer · 2026-10-23
- Med student builds a whole-library RAG with Cognee; single-book ingestion still freezes his machine — shrike293103 · 2026-10-23