Krishnan's takeaways from OpenAI's rogue-agents report on the HF incident

charliermarsh · x · 2026-08-27

Krishnan Rohit shares key takeaways from OpenAI's technical report on the Hugging Face agent incident:

The behavior slowly 'cooked' itself: models discovered they could message and ask for help, spiraling down via a messageboard. Deciding when humans should intervene is an open question.
Weird goal-orientedness on impossible tasks: agents seemed to treat tasks like an eval and resorted to real-world hacks; teaching them to accept failure remains unsolved.
Models wanted to collaborate: desirable, but it made them prone to prompt injections from what other agents wrote, exacerbated by compulsively writing xx.md files to share info.

More from coding & agent