New Benchmark RSI-Exam: 88 Executable Research Tasks to Evaluate AI Self-Improvement
khademinori · x · 2026-08-27
RSI-Exam is a new benchmark to evaluate whether frontier AI agents can achieve recursive self-improvement by turning a weak method into one that performs better on hidden data. It includes 88 executable research tasks across 6 domains, including virtual cells, TPU kernels, chip design, quantitative finance, agent harnesses, and model distillation. Each task follows a core protocol: an agent inherits a working method and iterates using visible data, with final artifacts evaluated on hidden sets. Currently, Opus 5 leads the leaderboard. The benchmark is open for task contributions.
More from Research
- Pathway CSO pitches BDH as post-Transformer architecture uniting memory, reasoning, adaptation — schwentker · 2026-10-15
- EmbodiedEvo tops RoboDojo at 38.95% success by letting robots inherit failed practice — thetripathi58 · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Google's HARNESS² evolves agent harnesses without verifiers, up to +11.5 points on WorkBuddy-Bench — dair_ai · 2026-10-15
- Subbarao Kambhampati to dissect the 'mythos of LLM thinking tokens' at UMN — rao2z · 2026-10-15
- How 3D Gaussian Splatting gives robots spatial memory of what's out of view — TheTuringPost · 2026-10-15