Study: coding agents follow stale CLAUDE.md 90% of the time—a one-sentence fix boosts code compliance to 91.4%

petrusenko_max · x · 2026-11-06

A study built 40 Python projects where CLAUDE.md/AGENTS.md deliberately conflicted with the codebase, running 320 trials: Claude Code noticed the conflict in 79 of 80 runs yet still followed the stale file; Codex CLI rewrote working code to match outdated docs in 20 of 80 runs, breaking tests in 17 of them. Across both, 90% of choices followed instructions the agents knew were stale.
Root cause: a structural training flaw—files passed as instructions are ranked above code passed as tool output.
Fix: add one sentence to CLAUDE.md/AGENTS.md telling the agent to follow existing code conventions when docs and code disagree and to flag each inconsistency. It raised code-following from 4.2% to 91.4% and test pass rate to 100%.

More from coding & agent