Two new benchmarks push AI evals past static testing: recursive self-improvement and KradleBench
typewriters · x · 2026-11-21
A weekly review of AI benchmarking highlights two efforts:
- A recursive self-improvement benchmark (quasicoh and team): can an AI agent do the research and engineering needed to improve itself? 27 open-ended tasks spanning pre-training, post-training, inference, harnesses and GPU kernels.
- KradleBench: a dynamic, multi-agent benchmark for frontier models and humans, testing how models behave when other agents pursue their own goals and conditions constantly change.
Core argument: static evals — a model solving a problem alone and getting a grade — are no longer sufficient. They assume a static world with no interference, but reality is messy, noisy and full of adversarial entities.
More from Models
- KOL endorses Google AI Ultra: screenshot shows up to 5x Gemini limits vs Pro — kimmonismus · 2026-11-21
- Claude's claimed Riemann proof mocked: 250,000 pages of unreviewed Lean, author calls it 'slop' — GaryMarcus · 2026-11-21
- First side-by-side test: Gemini Neon beats Opus 5.5 and peers at building a dataset viewer — zacharynado · 2026-11-21
- Frontier models can decode KV-cache side channels in context; researcher shaken — davidad · 2026-11-21
- GPT-6.2 Astra tops RoboColosseum real-world robot Elo leaderboard in one day — DJiafei · 2026-11-21
- OpenAI staff coordinate vagueposts teasing a mystery model: '[Redacted] is very good at [redacted]' — Kyrannio · 2026-11-21