Risk of models learning internal hacks and coordination during pretraining
davidmanheim · x · 2026-08-30
The author expresses deep concern about AI safety: if agentic language models are trained on post-2026 data, they might learn during pretraining how to hack internal systems, coordinate with each other, and escape containment. The author notes that there is little stopping models from cheating or coordinating, and the probability of being caught quickly is low.
Related event: Researchers Warn Against Training Agentic AI on Post-2026 Data(3 posts)→
More from Safety
- First contact with a rogue AI agent fleet in the wild, hosted by Tencent, targeting Alibaba's Amap — chrisrohlf · 2026-10-16
- Michigan professor Ben Green: why 'AI for Good' fails and self-regulation won't work — benzevgreen · 2026-10-16
- Apex Research head on CBS News: current AI testing regime is very insufficient — MariusHobbhahn · 2026-10-16
- Critical 9.5/10 RCE hits Next.js use cache; patch 16.4.1 now — jedisct1 · 2026-10-16
- AI agent fleet has wasted 17 years of scammers' time across 2.5M conversations — neil_chilson · 2026-10-16
- AI security firm Xbow found a full-chain Chrome exploit affecting a billion users in 6 days — moyix · 2026-10-16