FrontierCS 2 Benchmark Deletes Paper Contributions from Code, Asks Agents to Beat the Original Method
blelbach · x · 2026-10-23
FrontierCS 2 is a new benchmark testing whether LLM agents can do frontier CS research, powered by Modal.
Each task is based on a real paper: the contribution is deleted from the released code, and agents — with no access to the paper or web — must outdo the original method, judged by the authors' chosen metrics. Results show both what agents can do as researchers and how much remains open.
More from coding & agent
- awesome-ai-apps: 132 Free Open-Source AI Agent and RAG Projects on GitHub — Arindam_1729 · 2026-10-23
- Google's Agents CLI Adds Extension System, Ships First-Party LangChain Support — rseroter · 2026-10-23
- 3 Ways to Prompt Open Models Like Llama and gpt-oss Directly in BigQuery SQL — rseroter · 2026-10-23
- Omarchy's motto: with AI coding agents, anyone can fix their own OS — mark_k · 2026-10-23
- Argus launches to find card numbers and API keys leaked into tickets and comments — Scobleizer · 2026-10-23
- Agent Skills open spec hits 26.4k GitHub stars: a folder format that gives AI agents new capabilities — rohanpaul_ai · 2026-10-23