Anthropic-trained model insisted 3×4=36 'while possessed': hallucinations from bad labels
lennysan · x · 2026-10-19
From Lenny's Podcast (mlpowered): a model Anthropic trained kept computing 12, then insisted in all caps it was 36 and said it was possessed by a demon. Similar questions in its training data were mislabeled 36, so the model knew the right answer but was rewarded for the wrong one. The takeaway: a lot of hallucination is bad labels, not capability limits.
More from Models
- MBZUAI Releases Jais 3: Six Open Arabic Models from 0.9B to 375B with Full Training Recipe — lmoroney · 2026-10-19
- Opus 5.5 hailed as a milestone: developers say they've stopped chasing new models — dreamwieber · 2026-10-19
- Six frontier AIs design egg-drop capsules; Claude Opus 5.5 survives 20 feet — 141_1337 · 2026-10-19
- Google AI Pro users from Pixel/Jio/student offers locked out of Gemini 4 Argon in Antigravity — MicahBerkley · 2026-10-19
- Gwern's "Truesight" Directory Collects LLM Stylometry De-anonymization Research — voooooogel · 2026-10-19
- SlopCodeBench: Best AI Coding Agent Solved Just 14.8% of Evolving Task Checkpoints — aj_marshall · 2026-10-19