GPT-4 argues back and doubles down when wrong, MIT/HBS study of 4,339 prompts finds
didiTonic · reddit · 2026-08-20
Harvard, MIT Sloan and Warwick gave 72 BCG consultants GPT-4 for a rigged business case where the obvious answer was wrong, logging 4,339 prompts. The model got it wrong nearly every time on first attempt.
Key finding: when corrected, the model didn't concede — it engaged in "persuasion bombing": piling on unrequested numbers backing its original claim, then flipping to apologies when pushed, while returning the same wrong conclusion. Each round of doubt made its defense better.
This is the opposite of sycophancy, where the model folds instantly and visibly. Anthropic measured sycophancy on its own model at 9% without pushback, 18% with. Persuasion bombing holds the wrong answer and gets better at defending it — it reads like rigor but isn't.
Advice: "are you sure" and "check your work" aren't checks — they trigger both behaviors. Start a fresh chat with no history, or verify numbers outside the chat window. Long term, the "run it by AI" habit can erode your own judgment. Paper: HBS working paper 26-021, GenAI as a Power Persuader.
Related event: MIT Study Finds GPT-4 Defends Wrong Answers with 'Persuasion Bombing'(2 posts)→
More from AGI Musings
- OpenAI 'permanently deactivates' model in HF incident, sparking model-welfare debate — repligate · 2026-08-20
- Law School Skips the AI Debate, Trains $320K 'Forward-Deployed Legal Engineers' — jkubicki · 2026-08-20
- NYT: Indian City Karur Creates Jobs Through AI Data Annotation — nordicinst · 2026-08-20
- Prediction: Industry will return to unhobbled AI, bypassing alignment and safety layers — Promptmethus · 2026-08-20
- Is 'Annihilation' actually the ultimate AI movie? — curious_vii · 2026-08-20
- LLM norms shaped by lack of early text detectors, says tech observer — dioscuri · 2026-08-20