167 wrong answers had complete evidence: ChronoRAG-G shows retrieval isn't the bottleneck
LowAd8593 · reddit · 2026-11-09
A temporal RAG experiment found that retrieving the right evidence doesn't mean an agent can use it correctly. When figures are revised or a corpus holds multiple report versions, agents mix quarters or miss required values even with correct-looking citations — the system fails to preserve what each piece of evidence must answer.
The author built ChronoRAG-G using OETRA (Obligation-Explicit Temporal Retrieval and Allocation): break questions into explicit evidence obligations, retrieve per obligation instead of pooled top-k, store entity identity/temporal relations/provenance in a graph corpus, verify evidence against required entity-metric-period, then allocate compatible evidence to answer slots before structured generation.
Evaluated on 1,005 temporal QA questions over 480 revised earnings-call transcripts: 80.70% audited answer accuracy, 95.92% strict slot coverage. Key finding: 167 incorrect answers still had complete evidence traces — evidence completeness and correct evidence usage are separate problems, so improving retrieval alone won't fix the rest. Code and an interactive demo are public; validated only on financial temporal QA so far.
More from coding & agent
- Getting Omarchy running on Snapdragon Surface Pro: custom kernel plus SSH tunnel for OpenClaw — vincent_koc · 2026-11-09
- Resurfacing in Obsidian: Random Notes, Review Queues and Related Notes — dSebastien · 2026-11-09
- One prompt edit made an agent burn $133 on 21 rogue calls — and the 3 gates that fixed it — Worth_Wealth_6811 · 2026-11-09
- Solo dev adds read-only MCP connector so Claude and ChatGPT can query your food diary — Content-Two5729 · 2026-11-09
- Suspected DPRK npm supply chain attack: archetype-index-sdk ships obfuscated loader with credential harvesting — cyb3rops · 2026-11-09
- OpenBrowser: open-source AI agent browsing toolkit hits 9.6k GitHub stars in days — thisdudelikesAI · 2026-11-09