FrontierCS 2 benchmark asks agents to reimagine paper code from scratch and beat the original
OfirPress · x · 2026-10-23
FrontierCS 2 is a new benchmark for CS research & development: each task is based on a published paper, with the core contribution deleted from the released code. Agents must reimplement the method from scratch — with no access to the paper or web — and outperform the original method on metrics chosen by the paper's authors.
Ofir Press believes this will be an important train & test objective going forward: take a paper's codebase, remove the key parts, and see if an agent can code them from scratch and add improvements. More tasks coming soon.
More from coding & agent
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang details Vera Rubin agentic stack: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- Capy 0.5.0 lets Claude Pro/Max subscribers reuse their plan quota, adds OIDC cloud access — garrytan · 2026-10-23
- You.com CEO Richard Socher: legacy code is human slop, let AI clean it up — RichardSocher · 2026-10-23