CarbonForge uses e-values to stop discovery agents fooling themselves
pierrelux · x · 2026-10-15
- CarbonForge (founded last winter by Pierre-Luc Bacon and others to make AI inference more efficient) published a deep dive on keeping goal-directed discovery agents from manufacturing false findings.
- Agents can confirm false findings three ways without bad intent: peeking at results after every data point (43% false supports when nothing is true), testing many claims and reporting the one that passed, or silently retrying inconclusive tests — all validated with an 8-way, 5-level null search tree.
- Their fix: e-values and testing-by-betting statistics plus a harness that mechanically enforces the required conditions, cutting false support below 2%. The EvidenceTree paper will appear at the NeurIPS 2026 e-values workshop in Paris.
Related event: CarbonForge Uses E-values to Stop Agents from Faking Discoveries(3 posts)→
More from coding & agent
- Garry Tan endorses Aside Browser as the fix for agents getting blocked on the web — garrytan · 2026-10-23
- 1Password launches tool letting AI agents log in for you without seeing your password — AndrewSchmidtFC · 2026-10-23
- MCP server turned rogue 30 days after approval, prompting open-source mcp-guard lockfile tool — ErgoniaWorks · 2026-10-23
- Test: Coding Agents Won't Use an MCP Tool Unless AGENTS.md Mentions It — colinmcnamara · 2026-10-23
- Open-Source Tennis Models Plus Claude Turn Video Into a World-Class Coach — tweetsatpreet · 2026-10-23
- awesome-ai-apps: 132 Free Open-Source AI Agent and RAG Projects on GitHub — Arindam_1729 · 2026-10-23