CyberArena: live attack-defense benchmark drops agent capture rates to 35–52%

jiqizhixin · x · 2026-11-21

Researchers from 2077AI, UC Berkeley, UC Santa Cruz and UCSB introduce CyberArena, an arXiv benchmark where two cybersecurity agents attack and defend concurrently in isolated sandboxes—each must exploit the opposing service while patching its own and keeping it functional, with flags rotating each round.

The takeaway: static Jeopardy-style evals overstate real-world security agent capability; active defense materially changes measured success.

Original post →

More from Safety

Safety channel →