Agentic Search Systems Are Rewarded Only for Final Answers, Raising Faithfulness Concerns
burkov · x · 2026-10-16
Andriy Burkov highlights research on faithfulness in agentic search: LLMs are increasingly trained with RL to use search engines as tools, planning queries and reasoning over retrieved evidence. These systems ace QA benchmarks, but rewards typically depend only on the final answer being correct — leaving open whether intermediate reasoning steps are actually faithful to the gathered evidence or are plausible-looking post-hoc justifications.
Why it matters:
- Users rely on reasoning traces to audit how a conclusion was reached;
- Correct answers reached via faulty reasoning are unlikely to generalize or earn trust in high-stakes settings.
The work addresses the gap by first defining and measuring faithfulness in agentic search, breaking it into three distinct dimensions (thread truncated).
More from Research
- Survey: AI agent evaluation is getting realistic but still fails to measure safety and reliability — burkov · 2026-10-16
- COVA-SFT dataset trains models to solve text problems with visual chain-of-thought — yoavartzi · 2026-10-16
- USTC paper uses speculative sampling to cut latency in interactive video generation — ustc · 2026-10-16
- Tsinghua's CoTranslator distills reverse CoT to boost 4B model math scores — Tsinghua · 2026-10-16
- Nobel laureate Jennifer Doudna on AI's role in science in Big Think interview — ChiaraStellata · 2026-10-16
- DIY benchmark ranks 18 models on subjective judgments, GLM-5.3 tops consensus match — arkuto · 2026-10-16