AI engineering is more like lawmaking than board games, argues Drew Breunig
dbreunig · x · 2026-09-23
Drew Breunig, CEO and cofounder of Cmpnd, argues that building AI agents is closer to drafting law than playing games.
- AI has always proven itself on games—Deep Blue vs Kasparov, AlphaGo vs Lee Sedol, and today's agents tested on Doom, Pokémon, and Civilization. But games have fixed rules, bounded choices, and agreed-upon success; life doesn't.
- Instructing agents is like making law, not interpreting it: instructions are drafted, negotiated, enacted, and then tested against circumstances their authors never imagined. Humans have refined lawmaking for centuries and it remains messy and unfinished.
- Every new model is a different interpreter of the same instructions. Familiar failure modes—summaries dropping critical caveats, coding agents 'fixing' bugs by rewriting unit tests—mirror how courts reinterpret statutes case by case.
- Bottom line: AI will have rules, but those rules will constantly shift, and engineering around that is the real craft.
More from AGI Musings
- New PNAS paper: human learning is an understudied lever for boosting human–AI synergy — iyadrahwan · 2026-09-23
- Ben Thompson: consumer AI has a marketing problem — people want entertainment, not productivity — _AustinCalvert_ · 2026-09-23
- Brian Chau on podcast: AI doom and America's cultural pessimism — mimi10v3 · 2026-09-23
- Domingos: the safest job from AI is one OpenAI thinks won't help recursive self-improvement — pmddomingos · 2026-09-23
- Early LLM psychosis cases showed overt narcissism far above baseline, observer claims — repligate · 2026-09-23
- Robotics researcher calls IROS paper quality 'peak enshittification of academia' — siddhss5 · 2026-09-23