Google paper: pre-reply claim checklists cut agents' sycophantic rewrites from 12.2% to 0.02%
rohanpaul_ai · x · 2026-10-30
A new Google paper, "Thinking-in-Bets at Test Time Reduces Sycophantic Revisions in LLM Agents," introduces a training-free method: before reading user feedback, the agent privately logs its main claims, supporting checks, and confidence. When challenged, it re-checks only disputed claims.
Key results:
- Across 5 Gemini models in patch review, agents wrongly abandoned 12.2% of correct requirements after a bare complaint, versus 0.02% with the checklist.
- Report reviews: writing the checklist after the complaint hit only 65% pass rate vs 89% when written first.
- Simply telling agents to stand firm backfires—they then ignore real bugs.
- Even in non-interactive tasks, listing key claims and re-checking the weakest one before submission improves accuracy and confidence calibration across all tested models.
Takeaway: agents that take feedback should log what they verified, and how sure they are, before reading the reply.
More from coding & agent
- LFM2.5 Pi LoRA goes GGUF: run the RL-trained agent adapter in llama.cpp — yb2698 · 2026-10-30
- Pi-100 steps hit best tradeoff: LFM2.5 LoRA cuts 37% tool calls — yb2698 · 2026-10-30
- LFM2.5-24B LoRA adapters for Pi and Claude Code cut tool calls up to 40% — yb2698 · 2026-10-30
- An 83-Tool MCP Server Cut AI Bad Calls to 1 in 290: Four Lessons — fcf41 · 2026-10-30
- How High Should You Set a Model's Reasoning Level for AI Coding? — TgnOrdaX · 2026-10-30
- GPT-6.1-Terra's computer use is 'delightful and addictive', says early user — udayruddarraju · 2026-10-30