Developer finds data flaws in frontier benchmark ALE, releases fixed subset

A developer auditing the raw data of the frontier benchmark Agents' Last Exam (ALE) from UC Berkeley's Dawn Song team found issues, detailed in a new blog post. They ported the Linux subset to the Verifiers framework, releasing a corrected ALE-Gold subset.

2026-09-18 ~ 2026-09-18 · 3 related posts

• Porting Agents' Last Exam's Linux CLI subset surfaced benchmark defects, yielding ALE-Gold — dejavucoder · 2026-09-18
• Frontier benchmark data audit finds issues; porting Linux CLI subset to Verifiers v1 — dejavucoder · 2026-09-18
• New blog finds data issues in a frontier benchmark, ports Agents' Last Exam CLI subset to Verifiers v1 — dejavucoder · 2026-09-18