Nightly agent-learning job silently did nothing for a week while logging success every night
ChemicalExcellent463 · reddit · 2026-10-27
The author shares a self-built workflow and its failure modes: since mid-September, a nightly job reads their Claude Code and Codex sessions (focusing on places where they corrected an agent), turns them into one-line lessons for later sessions, with a review queue. Each lesson needs an exact session quote, verified by a second model. A bug let drafts skip that check, and a missing settings file left the job unable to send anything—yet it logged success nightly for a week before the author noticed, then discarded everything from that period. Tally so far: 7 approved, 44 rejected, 111 pending; automating the writing mostly meant more reading. Measuring impact is harder: the "run tests before saying done" lesson went from 22/119 skipped-test sessions to 4/12—not a win. The open question: how do you verify a lesson changes behavior rather than just getting shown again?
More from coding & agent
- Box explains its Deep Agent architecture: parent-child agents over a LangGraph orchestrator — LangChain · 2026-10-28
- How Box built enterprise content agents on Deep Agents with a parent/child architecture — LangChain · 2026-10-28
- How do you inject instructions mid-run? Dev struggles with stage summaries agents don't trust — tinker_bob · 2026-10-28
- Harrison Chase to demo agents that catch their own mistakes and write the eval and fix — LangChain · 2026-10-28
- Parallel launches Parallel Decisions API at $0.001 per request with 5,000 free calls — schwentker · 2026-10-28
- Together breaks down the MIGHT stack: composable open-source AI from models to MCP tools — togethercompute · 2026-10-28