From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation
Zhijun Gao, Jing Chen
cs.SE, cs.AI, cs.HC
2026-08-20
Across 557 sessions and 33,097 PRs, 60.5% of agent doc work hits instruction files and notes; API refs get 1.3%, and read-then-code almost never follows.
Documentation research still assumes a human reader: someone who gets lost, asks a colleague, and hunts through API pages for a task. A growing share of open-source changes is now written by coding agents that read the repo, run commands, and open pull requests. Guidance on "agent-friendly" docs (clear headings, runnable examples, llms.txt) is already circulating. It is built on how agents ought to behave. Almost nobody has measured what they actually open.
Zhijun Gao and Jing Chen at Peking University pin that question to traces. SWE-chat supplies 557 real agentic sessions and 94,813 development events, 3,033 of them documentation interactions. AIDev supplies 33,097 agentic pull requests and 690,260 classified file-level changes. The study asks what agents do with docs, what happens around those events, and whether the code-doc loop runs both ways.
The two datasets answer different questions and are never pooled. SWE-chat is process. The authors parse four incompatible transcript formats into a 20-symbol event alphabet, classify documentation by path rules into 15 types, then send 54% residual paths through a language model (500 paths covering 98.4% of ambiguous events). The large agentworkingnote category came out of that residual; it was absent from the initial scheme. Each documentation event is coded for interaction type (Discover, Search, Read, Edit, Create), trigger (four-event lookback), and stage (orientation, implementation, verification, debugging, delivery). Some agents hide file edits inside shell applypatch here-documents; without parsing command text, one family would have registered zero documentation events.
AIDev is artefacts. On a curated subset of repositories with more than 100 stars, they mark whether a PR touches code, documentation, or both, then recover order in multi-commit PRs.
Statistics are conservative. Events nest in sessions, PRs nest in repositories. Primary intervals come from a cluster bootstrap (2,000 resamples of whole sessions or whole repositories). Downstream actions within three events of a consultation get a session-clustered logistic GEE that adjusts for stage, position, session length, and agent family. Sampling oversampled minority agents, so headline shares appear under event weighting, session-equal weighting, and reweighting to the corpus agent mix.
The instrument only sees repository-local, path-based documentation. External API websites in a browser, knowledge in model weights, in-source comments, and context files loaded at session start (unless later re-read) are out of scope. Instruction-file counts are lower bounds on exposure.
The type mix does not look like the documentation literature.
| Document type | Events | Share |
| Agent instruction files (AGENTS.md, CLAUDE.md, SKILL.md, ...) | 1,074 | 35.4% |
| Agent working notes (plans, thoughts/, brainstorms) | 760 | 25.1% |
| Task / requirements | 301 | 9.9% |
| Configuration | 205 | 6.8% |
| README | 197 | 6.5% |
| Nine classical technical-doc genres combined | 323 | 10.6% |
| API reference | 40 | 1.3% |
| Troubleshooting | 11 | 0.4% |
Agent-facing artefacts are 60.5% of events (cluster CI 53.9-66.5%). Agent reweighting drops that to 55.1%; on consultation alone it sits at 50.1%, so majority status depends on the weighting. API references are 2.3% of observable in-repo consultations. Instruction files see about 27 times more interactions than API references.
The adjacent read-then-edit-code pattern is almost missing: P(edit code | read doc) = 0.002, three cases in 1,328 reads. A read is more often followed by another read (0.270) or by reasoning (0.245). Inside a three-event window, running a test has lift 0.23 (cluster CI 0.08-0.45; adjusted OR 0.39) and building has lift 0.15 (OR 0.25). Both hold before and after adjustment. Documentation creation (lift 1.67) and code editing (lift 1.05 / OR 1.33) flip with adjustment; the paper marks those couplings unresolved.
Triggers: 70.2% self-initiated, 7.5% failure-driven, a 9.3x gap. Of 2,034 failure episodes, reading documentation is the first recovery move 109 times (5.4%). Reading code (31.0%), retrying (19.9%), and editing directly (15.3%) are all more common. Troubleshooting docs appear 11 times in the whole corpus.
Agents write documentation nearly as often as they read it: 1,401 production events versus 1,615 consultations, ratio 0.87. On AIDev, 41.5% of agentic PRs change documentation (cluster CI 35.8-45.4%). Where order is observable in multi-commit PRs, code is touched first 4.7 times more often than documentation (82.5% of different-commit cases). The most-edited documentation files include AGENTS.md (692 PRs), CLAUDE.md (362), and copilot-instructions.md (287): agents edit the instructions that will condition the next agent.
The linear journey Discover-Retrieve-Interpret-Apply-Validate-Update does not fit. The traces look like a two-lobed cycle: a consultation lobe that loops through reading and reasoning, a production lobe that writes documents, a weak and specification-sensitive coupling between them, and zero Validate events in the tool-call traces.
For maintainers the implication is narrow and sharp. Finite documentation effort spent on AGENTS.md and CLAUDE.md will meet agents far more often than equivalent effort on API reference prose. Follow-reference is unattested; reads come in runs. Self-contained, locally retrievable files match the observed behaviour better than richly cross-linked manuals.
A second surface is already in the repo and mostly untracked: agent working notes (plans, thoughts/, brainstorms) are 25.1% of interactions and persist as files. Quality metrics, review checklists, and hygiene tools have no category for them. The paper measures volume, not staleness or inconsistency.
Actionability and verifiability remain legitimate design goals. They do not follow from this behaviour. No sequence treats prose as an oracle against which code is checked. If that check needs to become observable, executable artefacts (doctests, schema contracts) are the intervention to test, not a finding of this study.
The largest hole is the classifier. The agentworkingnote category, 25.1% of events, rests on language-model labels for 500 ambiguous paths. No human validation. The authors call dual coding of 200-300 events with kappa the necessary next step. The exact share is provisional; the qualitative claim that a large, previously uncategorised class of agent-authored notes exists is more robust, because it is visible in the raw paths.
The instrument is path-based. Docstrings, inline comments, and API sites opened in a browser are invisible, so absolute rates are lower bounds. Stage labels stick in debugging after a failure signal, which is why the paper only claims documentation is not confined to orientation, and refuses to treat the 54.4% debugging share as a precise allocation.
SWE-chat is opt-in telemetry. About 87% of the corpus comes from one agent family (Claude Code is 83.8% of labelled sessions). Cursor shows 0/11 sessions with documentation events; the paper treats that as an extraction-coverage warning. Agents that hide file work inside shell here-documents register zero events unless those paths are parsed. Per-agent rates are not comparable behavioural estimates.
A near-zero adjacent transition does not rule out longer-range influence, or influence inside reasoning text the instrument cannot see. The three-event lift is the mitigation, and even then the code-edit coupling stays unresolved. Documentation-first recovery has the highest point estimate of resolution (7/11 = 63.6%), with an interval of 35.4-84.8% that overlaps every other strategy. The paper declines to rank recovery tactics.
The two datasets are different populations. Private codebases, non-CLI agents, and a practice that is about two years old can all move that 60.5%. What is expected to persist is the category, not the exact share.