Decision models match Gemini for RAG conflict typing at 1/39 the cost and 6x speed
parvkhatri · reddit · 2026-11-04
This paper tackles RAG knowledge conflicts: the right answer depends on why sources disagree (multiple valid answers, opinion, outdated info, or falsehood).
- 11 typed decision models from 7 providers were tested on the CONFLICTS benchmark (406 queries): 10/11 match or beat Gemini 2.5 Flash (61.3%), best scores 67.0%.
- Paid options cost 4–39x less and run 2.6–6x faster; end-to-end answers match the original LLM pipeline for 10/11 models.
- Caveat: Gemini 3.8 Flash hits 70.0%, beating 8 of 11 — but cascading the 25% least-confident queries to it ties it at 25–37% of the cost.
- Code, prompts and all outputs are open; every number is reproducible without API calls.
More from coding & agent
- Agent silently completed only a quarter of a 200-module build and stalled waiting for confirmation — Promptmethus · 2026-11-04
- ChatGPT Agent mode can scan LinkedIn and Indeed and build a tailored job-application spreadsheet — nikola_mr64990 · 2026-11-04
- Blogger auto-applied to 500 jobs via ChatGPT and got 12 interview calls in 24 hours — nikola_mr64990 · 2026-11-04
- MCP servers with hundreds of endpoints: a search-spec-call pattern that keeps context small — sandeepK1125 · 2026-11-04
- Tencent's Hy4 agent self-trained a smaller model that beat Tencent's own baseline — heyshrutimishra · 2026-11-04
- Butterfly traces multi-agent failures with graphs, 62.5% accuracy vs 53.8% for log prompting — JeremyCMorgan · 2026-11-04