FrontierCS 2: benchmark makes AI agents beat published CS research blind

arena · x · 2026-11-18

UC Berkeley PhD candidates MangQiuyang and lihanc02 joined Arena to introduce FrontierCS 2, a benchmark challenging AI agents to outperform published CS methods without seeing the paper or web access. They argue reliable benchmarks require more than good tasks: defending against reward hacking and robust eval infrastructure matter equally for open-ended research tasks. The discussion covers evaluation exploits like bypassing RL with search and reusing cached graph-algorithm results, and why human research judgment still matters.

Original post →

More from coding & agent

coding & agent channel →