Benchmark Almanac indexes 3,322 AI benchmarks across 1,746 leaderboards in one repo
karinanguyen · x · 2026-11-14
Brian Zhou launched Benchmark Almanac, reportedly the most comprehensive index of AI evaluation benchmarks to date.
- Tracks 3,322 benchmarks and 1,746 leaderboards
- Aggregates 522 launch posts, 346 system cards, and 1,233 open-source GitHub repos
- Aims to solve the fragmentation problem of keeping up with the ever-growing eval ecosystem
Karin Nguyen amplified the launch with an emphatic "evals evals evals," reflecting how central evaluation has become to the field.
More from Research
- SGLang Summit talk details battle-tested NVFP4 4-bit RL across thousands of GPUs — ericzelikman · 2026-11-14
- Decision models tested: training buys speed and cost, not reasoning back — someone_12443 · 2026-11-14
- Ropedia open-sources Xperience 2: 1,589 3D scene scans and 131 hours of egocentric demos for physical AI — liuziwei7 · 2026-11-14
- Open-Source Self-Driving Textbook on GitHub Builds an Autonomous System Chapter by Chapter — 4310sy · 2026-11-14
- RobotAgentBench: hybrid Codex-InternVLA-A1.5 system averages 48.5 on household agent tasks — jiqizhixin · 2026-11-14
- NetHack transpiler contest: serteal first to hit 100% on all public test sessions — davidbau · 2026-11-14