Only 32% of project rules reach Claude Code; docs lift it to 70% — open benchmark context-recall

AttitudeLatter7306 · reddit · 2026-10-28

Segmently released context-recall, an open benchmark measuring how many project rules actually reach a coding agent.

Key findings

Extracted 75 rules (6/task) from 12 real past tasks; of 70 traceable rules: 24 live only in comments/error messages, 13 enforced by tests/types, 22 implied, 11 absent from code entirely
57/70 rules have no test that fails when broken; across 15 broader tasks, 1/3 of needed rules were written nowhere
Code-indexing tools (Serena, Trail, Graphify) scored 24-29% without docs, 46-47% with — no better than plain Claude Code, since these rules aren't in the code

Benchmark results (headless Claude Code + blinded LLM judge)

No project docs: 32% of rules stated in planning briefs
Docs in repo: 54%
Segmently's setup (short critical-rules doc per module, retrieved by meaning, read fully before planning): 70%

Open source: context-recall (MIT) runs as a Claude Code plugin; snapshot repos before real changes, run any tool as an arm, judge blind, compare.

More from coding & agent