Epoch AI Reviews 4 More Benchmarks: Only GraphWalks Verified, OSWorld 2.1 Flawed
xeophon · x · 2026-10-24
Epoch AI published four new benchmark reviews: only one is Verified (GraphWalks), three are Flawed (OSWorld 2.1, CritPt v1, CMT-Benchmark v1.0.0), and four more lacked sufficient information for a verdict (Remote Labor Index, TasteVal, Chartography, Surface Evolver Bench).
Notably, CritPt v1 was previously rated "Not enough info" before being marked Flawed. Epoch says reviewer capacity is the main bottleneck to reviewing more benchmarks and is recruiting reviewers. Results are CC-BY licensed at epoch.ai.
More from Research
- Ex-OpenAI's Yaroslav Bulatov on why backprop should go: it's the memory wall — yaroslavvb · 2026-10-24
- UBC NLP lands 21 papers at EMNLP 2026 spanning culture, RAG, and agents — mageed · 2026-10-24
- Terminal-Bench exposes NVIDIA Nemotron 3.5 near-zero score vs Kimi K3.1's 65.7% — JJitsev · 2026-10-24
- Kimi K3.1 Pro reportedly first Chinese model to solve 20x20 nonograms — fairydreaming · 2026-10-24
- Self-Evolving Benchmarks: Multi-Agent Evolutionary Search Auto-Generates Challenging Tool-Use Tasks — MichaelD1729 · 2026-10-24
- Colossal didn't de-extinct the dire wolf — they gene-edited gray wolves to look like them — shae_mcl · 2026-10-24