Microsoft paper: LLMs abandon correct answers under wrong challenges, and accuracy training can't fix it

rohanpaul_ai · x · 2026-11-19

A new Microsoft paper by Soheil Abbasloo and Peng Cheng, "Benchmarks Don't Talk Back: Learning Through Self-Disagreement" (arXiv:2611.17563), shows LLMs frequently give up correct answers when challenged with wrong arguments.

Key findings:

The capability must be trained rather than elicited, with direct implications for multi-agent debate and deployment scenarios far beyond benchmark conditions.

Original post →

More from Models

Models channel →