Vals AI launches RSI Index benchmarking whether models can autonomously do LLM R&D
test_tm7873 · x · 2026-10-29
Evaluation firm Vals AI introduced the proprietary Vals RSI Index, which asks a single question: can a model do the research that builds the next model — i.e., progress toward Recursive Self-Improvement.
- What's tested: RSI is decomposed into autonomous research tasks spanning the full training pipeline — architecture, algorithms, data acquisition and selection across pre/post-training, eval design, and systems engineering. Each task runs under a fixed compute and time budget; the model researches, experiments, and submits one artifact for grading.
- Harness: Models run in off-the-shelf agent tools — Claude and GLM in Claude Code, GPT in Codex, Gemini in Antigravity, Kimi in Kimi Code, Qwen in Qwen Code, Grok in Grok Build — at max reasoning effort until the time budget is exhausted.
- References: Scores are graded against published work, human references, and frontier-model outputs.
The index turns "when will RSI happen" into a trackable leaderboard; updates are worth watching.
More from Research
- UC Berkeley's RaRE Shows Judge Ensembles Beat Single Judges in Rubrics-as-Rewards RL — YutongBAI1002 · 2026-10-29
- BallotBench pits 9 frontier AI agents against the 2026 midterms; GPT 6.1 alone picks Democrats — goyal__pramod · 2026-10-29
- 3DV2027 Decisions Go Out Today; Author Rebuttal Deadline Extended to Nov 2 — CSProfKGD · 2026-10-29
- Samaya's 3B-Parameter Sterling Beats Claude Fable 5.1 on Finance Benchmark at 11x Lower Cost — simonguozirui · 2026-10-29
- Discussion: Can language model critics become too predictable "deterministic parrots"? — IgorBrigadir · 2026-10-29
- Training a Small OSS Model on Factorio Tasks Could Be a Great Generalization Study — rmcwhorter99 · 2026-10-29