EFF's AI-Progress-Metrics: A Long-Running Tracker of Benchmarks Across AI Subfields
robseamans · x · 2026-10-14
Robert Seamans shares EFF's AI-progress-metrics project, a pilot effort collecting problems, metrics and datasets from AI research literature to track progress across subfields (created by Peter Eckersley and Yomna Nasser).
- Usable to see how specific subfields or AI/ML overall are progressing, report new results, or find problems needing new datasets/metrics
- Merges data from machine vision benchmark trackers, computer chess history, speech recognition WER collections, visual QA surveys, and algorithmic progress studies
- Received input from Dario Amodei, Helen Toner and many others; supported by the Open Philanthropy Project
More from Research
- Fingerprinting Leaderboard benchmarks model provenance as copying takes minutes vs millions to train — omarsar0 · 2026-10-14
- Is today's agent memory stack just a workaround for model architecture? — Sun-Tzu-Maybe · 2026-10-14
- Persona experiments on Talkie, a 13B LLM trained only on pre-1931 text — dhadfieldmenell · 2026-10-14
- HKU and Tencent open-source Multics, a multi-player world model that trains on 4 players but scales to 10 — Scobleizer · 2026-10-14
- SAP paper teaches LLM agents to predict their own failure (AUROC 0.85) — SAP · 2026-10-14
- CUHK's Orchestra robot framework completes 7.35-step subtask sequences — CUOHK · 2026-10-14