LLMs adopt revisionist history at 81-97% rate under prompting, EMNLP study finds
ZhijingJin · x · 2026-10-26
A team led by Zhijing Jin presents 'Preserving Historical Truth' (arXiv:2602.17433) at EMNLP, auditing how LLMs handle contested historical events.
Key findings
- Mistral and DeepSeek are the most revisionist models tested; explicit revisionist prompts push all tested LLMs to produce revisionist narratives at 81-97% rates, exceeding 96% for Mistral-7B and DeepSeek-R1.
- Under neutral prompts models generally align with facts, but scenario matters: social media posts, debate arguments and article correction show the highest non-factual rates, while fact-checking prompts are consistently safest.
Method
- HistoricalMisinfo benchmark: 500 contested events from 45 countries, each with factual and documented revisionist reference narratives.
- 11 real-world prompt scenarios (textbooks, museum labels, social posts, policy briefs, commemorative speeches, etc.) evaluated via an LLM-as-a-judge protocol.
Authors include Bernhard Schölkopf and Rada Mihalcea; paper, code and dataset are public.
More from Safety
- PHP WASM servers show no sign of PHP2SHELL impact so far, patch in progress — MickeySteamboat · 2026-10-26
- Apple's week-long curl/git CA bundle lag sparks developer security debate — kingslyj · 2026-10-26
- LLMs and the Law: training doesn't make a model a derivative work, FAQ explains — esrtweet · 2026-10-26
- OpenAI's Daybreak Blue isn't hard for individuals to get — just set up a hardware key 2FA — pvncher · 2026-10-26
- Research agenda for multi-agent swarms: 7 open alignment problems — ZhijingJin · 2026-10-26
- LiveOverflow digs into stealmybitcoin.com CTF, suspects PHP2SHELL challenge is deliberately unexploitable — matthew_d_green · 2026-10-26