Google & CMU paper: multi-objective harness evolution keeps 55% of security gains vs 0% baseline
rohanpaul_ai · x · 2026-10-15
A new Google and Carnegie Mellon paper, Harmony Harness, tackles a key flaw in self-improving agents: security gains vanish when the harness optimizes a single score like test pass rate.
- Harmony Harness evolves multiple harness versions along a Pareto frontier, each balancing passing tests vs. blocking exploits, using decomposition-based evolutionary search
- It separates general-purpose fixes from benchmark-specific adaptations and composes them per task at inference
- On unseen crash types in SecRepoBench, 66.7% of its fixes both worked and blocked the exploit (vs. 36.7% for the starting harness), and it retained 55% of security gains where the DGM baseline retained 0%
Takeaway: when agents improve themselves, track security as its own objective and validate on held-out tasks.
More from coding & agent
- OpenUI, an MIT-licensed open standard for generative UI, hits 10.9k GitHub stars — testingcatalog · 2026-10-16
- Agent registers and pays $24.95 for traffic school via Stripe Link, then refuses to take the course — jeff_weinstein · 2026-10-16
- Netlify Releases Beginner Tutorial on Building Apps with AI Gateway — thisiskp_ · 2026-10-16
- MCP Apps: Spring AI servers can now return rich interactive UIs alongside tool results — therealdanvega · 2026-10-16
- SDLC is becoming ADLC: Limestone's seven-step agent pipeline keeps engineers as approvers — alex_verem · 2026-10-16
- Huge opportunity in enterprise autonomous agents, but most companies point them at the wrong work — alex_verem · 2026-10-16