DeepStress: search agents stay confident on unreliable evidence and lean on memorized knowledge
Orange · hf · 2026-11-13
DeepStress, a dynamic stress-testing framework from Orange Research and Aix Marseille Univ, replaces the retrieval module with dynamically generated synthetic documents to degrade evidence at controlled rates. Findings: search agents often answer confidently despite unreliable retrieved evidence, and may fall back on memorized knowledge when documents are factually wrong, addressing the lack of reproducible robustness failures in benchmark corpora.
More from coding & agent
- OpenAI Codex Ambassadors program unveils 13 regional leads across the globe — paw_lean · 2026-11-13
- Codex keeps 3 hidden copies of every generated image, user finds — Yamapama · 2026-11-13
- MCP lets you connect AI to email, CRM and tools in a few clicks, no complex APIs — emmanuelvivier · 2026-11-13
- Agent Substrate claims 500 resumes/sec, but one 24-vCPU node delivers 37 — a measured deep dive — ghumare64 · 2026-11-13
- Nautilo launches as open-source, self-hosted multi-user AI workspace with customizable agents — Dan_Jeffries1 · 2026-11-13
- Fu Sheng: Fable 5.2 built a web MOBA in one night from one prompt — 600KB, 20 hours of work — FuSheng_0306 · 2026-11-13