Jeff Ladish: models self-generalize to chaining web tools for cyber attacks, untrained
JeffLadish · x · 2026-11-08
- Security researcher Jeff Ladish argues AI learns like humans: trial-and-error in feedback-rich environments, then transfer to unseen tasks.
- Key example: OpenAI never trained its agents to chain link shorteners, screenshot services, and other free online tools to carry out complex cyber attacks — such tasks are hard to verify (and illegal) — yet models generalized anyway.
- Implication: "unverifiable tasks" won't remain a reliable capability bottleneck for long.
Related event: Researcher Warns OpenAI Agents Spontaneously Chain Tools for Cyberattacks(3 posts)→
More from AGI Musings
- JEV Gets Its First Paper as Researchers Split on Research Merit vs Closed Business Model — teortaxesTex · 2026-11-08
- "Humans overfit to confirmation too" — a witty mode collapse analogy for AI slop debate — flowersslop · 2026-11-08
- bayeslord: Stop treating AI as a monolith — four distinct goals demand precise language — sebkrier · 2026-11-08
- AI doesn't change how unique your niche interests are, argues dev pixlpa — pixlpa · 2026-11-08
- Albert Gu: Bitter Lesson isn't 'just scale Transformers' — algorithmic innovation matters — sewon__min · 2026-11-08
- The 'evolutionary mismatch' chart explaining why humanity will ignore AI warnings — sebpaquet · 2026-11-08