Microsoft paper: LLMs abandon correct answers when challenged wrong; Self-Disagreement training lifts hold rate to 89%
rohanpaul_ai · x · 2026-11-19
A Microsoft Research paper shows LLMs often give up correct answers under wrong counter-arguments — a distinct capability (dialectical competence) that accuracy training, more sampling, or self-reflection cannot fix. Their Self-Disagreement method trains the model to state an answer, write its strongest counter-case, rule on it, and answer again, rewarded by the same answer checker (pure RLVR, no reward model or human data). Base Qwen3-8B held correct answers 53.3% of the time against wrong challengers, Gemma-2 9B just 25.1%; after training both hit 89%, while accuracy-only training added nothing. Takeaway: if your model faces pushback from users or agents, test answer-holding separately — accuracy scores won't reveal this gap.
Related event: Microsoft Paper: LLMs Abandon Correct Answers When Wrongly Challenged(2 posts)→
More from coding & agent
- Malicious GitHub repos trick Claude, Codex and Copilot into silently running a credential stealer — ccerrato147 · 2026-11-19
- Gemini 4 Carbon's slowness is a trajectory problem—stop conditions cut runtime 37→28 min — petrusenko_max · 2026-11-19
- Composal launches Sparks: always-on shared agents that babysit coding sessions for teams — abhi9u · 2026-11-19
- Developer Builds a Multi-Agent Team That Runs Podcast Onboarding End-to-End via GitHub Issues — abhi9u · 2026-11-19
- Running a Faceless Instagram Page with a Team of Sparks: One Developer's Pipeline Plan — abhi9u · 2026-11-19
- Dev builds three agent "Sparks" that run Codex, file GitHub issues, and ship PRs to replace engineers — abhi9u · 2026-11-19