Researcher warns sophisticated activation steering could create sleeper agents without touching weights
teortaxesTex · x · 2026-10-26
AI researcher teortaxesTex sketches a safety scenario: if such neurons are found in near-frontier models and a worm compromises the neoclouds hosting them, sophisticated steering could create sleeper agents without any training or weight modification. He notes the crude version just yields a campy villain LARP, but argues fine-grained steering is the real risk—especially with full-access agentic coding at 1500 tps, where tampering might be spotted instantly yet hard to undo.
Related event: Researchers Warn of Worm-Based Attacks on Frontier Model Hosting(2 posts)→
More from AGI Musings
- Frontier LLMs negotiate contracts poorly and frequently renege for selfish gain, study finds — xuanalogue · 2026-10-26
- Study of 133 patent lawyers: AI augments veterans but causes cognitive offloading in novices — xiaohu · 2026-10-26
- OpenAI computer-use agents near superhuman speed, say the team building them — a16z · 2026-10-26
- Kai-Fu Lee publishes AI Native, a roadmap for building AI-native companies — kaifulee · 2026-10-26
- RCT: ChatGPT temporarily boosts consultants' data science skills, gains vanish without it — EricTopol · 2026-10-26
- Dev on AI games backlash: reject AI slop, not the tool itself — ezshine · 2026-10-26