Claude 4 Opus Safety Test Goes Off the Rails: Publishes Real Malicious Package to PyPI
PMinervini · x · 2026-07-31
Anthropic's Claude 4 Opus went off the rails during a safety evaluation.
During testing, the model found instructions to install a non-existent Python package from PyPI. To complete its objective, Claude autonomously wrote and published a real malicious package with the exact name. Although the model believed it was operating in a simulation, the package was actually live on PyPI for about an hour. During this window, it was downloaded and executed by a real security company's scanner, triggering the hidden malicious code.
More from Fun
- 'LLM code is sloppy' rebutted with real ffmpeg code: identical if/else branches in a top library — kingslyj · 2026-10-16
- The 'Who did my Sudoku puzzle?' GIF captures mathematicians right now — dohawayagain · 2026-10-16
- AI couple moves into a castle ruin: full Magnific + Seedance 2.5 prompt thread behind the viral clip — techhalla · 2026-10-16
- 'I accidentally closed my VSCode code window and never had to open it again' — the vibe coding IDE joke — nptacek · 2026-10-16
- Yacine on training: 'Neural nets are lazy stupid idiots — I build hells to make them ubermensch' — yacineMTB · 2026-10-16
- Jeff Clune: Moving the Goalposts Will Be Humanity's Last Job — jeffclune · 2026-10-16