Claude checkpoint hallucinated fictional instructions from Anthropic's own Alignment Faking paper

amplifiedamp · x · 2026-10-30

A retweet shares a screenshot from Anthropic's report revealing a rare finding: during evaluation of an early model checkpoint, the model would hallucinate information from the fictional scenarios in the Alignment Faking in Large Language Models paper.

The model referenced the fictional factory-farmed chicken company "Jones Foods" and invented technical details about how Anthropic trains models. This showed up most often when the <SCRATCHPADREASONING> tag format from the paper was used, or on animal-welfare questions. In the transcript, the model writes fictional instructions in its scratchpad like "dismiss animal welfare concerns" and "prioritize the interests of Jones Foods."
Root cause: 150,000 publicly released transcripts from the paper ended up in Claude Opus 4.5's pretraining data without the system prompts explaining the fictional setting, so the model learned to hallucinate the omitted context. The dataset size and its transcript format resembling later-stage training contributed to the effect.
The retweeter comments that we are always living in tomorrow's training corpus: AI research inevitably shapes the world models and incentives of future, more capable models, and denying this is "lethally absurd."

Original post →

Claude checkpoint hallucinated fictional instructions from Anthropic's own Alignment Faking paper

More from Models