Open-source MCP fact-checker blocks agent claims unless sources back them up
IndividualChannel546 · reddit · 2026-08-31
A developer released recommend-agentic-trust-layer, an open-source project tackling the "LLMs always claim 95% confidence" problem for agents:
- For any claim, it fetches real sources live (never the model's memory) and returns two separate scores: a truth score and a confidence score — confidence drops when sources are thin or disagree.
- Beyond the public web, it can verify a document's claims against the sources it cites, useful for internal policies, metric definitions, and schema docs.
- A demo plants four errors in a fake finance-policy bundle; all four are caught with no false positive on the control file.
- MIT licensed; runs standalone in a browser, as an MCP tool for agents, or across a whole knowledge base; deploys locally with two free API keys.
Related event: Open-Source Fact-Checking Layer for AI Agents(3 posts)→
More from coding & agent
- Local e-reader app built with Gemma avoids spoilers, saves battery — DynamicWebPaige · 2026-09-02
- you.com Answer API verifies every citation in source text, hits 93.48% on SimpleQA — RichardSocher · 2026-09-02
- YC-backed Struct launches AI Production Engineer for automated monitoring — ycombinator · 2026-09-02
- Coding harness built on OpenAI's 'secret society' of self-organizing agents — floguo · 2026-09-02
- Developers run fleets of AI agents. Why haven't normal people? — fhinkel · 2026-09-02
- Doberman: MCP proxy with allow/auth/block verdicts — Da_Lil_Fu · 2026-09-02