Meta AI introduces AutoBenchmark for automated benchmark creation

Meta AI researcher Jason Weston introduced AutoBenchmark, where autoresearch agents auto-create benchmarks to evaluate themselves, finding the difficulty transfers to unseen models and that targeted human intervention helps when the research loop stalls.

2026-09-30 ~ 2026-09-30 · 3 related posts

• Meta AI's AutoBenchmark auto-creates benchmarks, finds human feedback gives big wins over agents alone — jaseweston · 2026-09-30
• AutoBenchmark: benchmark difficulty transfers to held-out models like Nemotron and Claude Opus-5 — jaseweston · 2026-09-30
• AutoBenchmark wraps up: human direction helps when autoresearch loops stall, plus Rebuttal Bench — jaseweston · 2026-09-30