Two Haiku agents + grep hit 77% line recall on ContextBench at ~1¢ per issue

Opening-Round5528 · reddit · 2026-11-18

CodeNib retrieves code context on ContextBench (500 tasks, 8 languages) with two Claude Haiku 5.5 agents and grep — no embeddings, no reranker — finding 77% of human-annotated relevant lines vs 61% for the next-best system, at 1¢ per issue. Agent one follows the issue with glob/grep/reads; agent two gets the issue plus a ≤2,048-token map of where agent one looked and hunts what was missed (callers, alternate paths, tests, config). The broad search has only 20.5% line precision; with an Opus repair agent on top, the system resolves 388/500 tasks, #1 among agent systems as of Oct 23 (task annotations were visible during development). Write-up, trajectories and cost receipts are public.

Original post →

More from coding & agent

coding & agent channel →