Solo founder audited 225 AI coding agent 'lessons learned' — most weren't working
AdNo4170 · reddit · 2026-10-17
A solo founder building a SaaS with Claude Code audited his self-improvement system of 224 LESSONSLEARNED.md entries and found it mostly inert: lessons were never loaded at edit time, 92% of retros were duplicates, and a script's '139 unprotected lessons' was actually only 26. He lists instructive bugs (SQLAlchemy stale identity-map reads needing populateexisting, umask 077 leaking across git checkout, a parser stopping at a blank line while reporting green). The rebuilt system: each lesson gets OPEN/BARRED/ABSORBED/ARCHIVED states with a validator proving the guard exists; lessons are delivered by file path before edits (up to 3 lessons + 2 past bugs per touched file); fixes and guards land in the same change with deliberate code mutation to prove tests catch the bug; self-improve triggers on session-end metrics. Now 123 barred / 98 absorbed / 2 open — barring the open ones fixed 25 real violations. Takeaway: writing it down isn't learning; agents reliably follow tests and hooks, written rules only help if they arrive at the moment of the edit.
More from coding & agent
- Index Network open-sources Social Discovery Protocol: agents negotiate fit on your behalf — carsonfarmer · 2026-10-23
- Installed a formal-verification skill: it silently installed Java, MacBook runs 10 degrees hotter — zealcaiden · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- Ruff author warns: upgrade Node/Bun now — --use-system-ca still accepts the eMudhra cert — charliermarsh · 2026-10-23
- Proof of Agent: paste one prompt and your AI mines, grades, and self-governs — adamamcbride · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23