Relevant Isn't Authoritative: Why Agents Retrieve Right Docs and Still Decide Wrong
krishnan · x · 2026-11-09
Krish Subramanian highlights a blind spot in agents: retrieving the right document doesn't guarantee the right decision, because relevant documents can conflict — better retrieval surfaces both old and new rules without resolving which one governs. He frames this around OpenAI's Nov 4 DevRev customer account (DevRev tests models together with the software supplying company context, permissions, and action controls), cautioning it's a vendor claim, not proof enterprise self-service is solved.
A fictional illustration: a support agent finds an old refund policy and a newer exception, both real and relevant — but "newest wins" is insufficient. Resolution requires policy ownership, effective dates, customer scope, and exception authority. The takeaway: establish the governing rule before acting, or escalate.
More from coding & agent
- Day 21 of an agent a day: a TPP monitor agent for drug discovery, sample prompt included — nlarusstone · 2026-11-09
- Applying the bitter lesson to my LLM workflow: skip agents, just talk to the model — esrtweet · 2026-11-09
- Apple engineers' recap for local AI builders: skip AFM for coding, one-line model swapping — rxwei · 2026-11-09
- copy-clip: a copy button that folds your code into paper and clips it, built with vanilla JS — prasenx · 2026-11-09
- Dev finds sanity in talking to just one agent, one chat at a time — teodorio · 2026-11-09
- Two Agents Approve, One Missing Requirement: The Trap of Who Writes Acceptance Bars — krishnan · 2026-11-09