ELITE evolves coding agents with zero task evaluations, lifting SWE-bench Pro solve rate from 60% to 86%
RobertTLange · x · 2026-11-21
Robert Lange highlights the paper "Evaluation-Free Recursive Self-Improvement for Coding Agents" (Ren et al., Jilin University & KAUST).
- Problem: self-improving coding agents typically pick harness edits via downstream task evaluation (evaluate-and-select), which is costly and overfits benchmarks.
- Method: ELITE (Explore-then-Integrate) starts each generation's edits from a common parent, inspects the resulting programs, and reimplements complementary changes into one coherent descendant — using only deterministic checks (does it compile, do tools run) inside the loop, no task evaluation.
- Results: on a SWE-bench Pro subset, the DeepSeek agent's solve rate rises from 60% to 86% (vs. DGM 86.7%, HGM 88.3%) with zero candidate evaluations and 8.8×/3.2× faster wall-clock evolution.
- Cross-model: the DeepSeek-evolved harness transfers to Qwen, averaging 73.3% on SWE-bench Pro, 88.5% on Verified, 69.3% on Polyglot.
- Ablation: dropping the basic validity checks erases the gains.
More from coding & agent
- mnemos: a self-curating agent memory plugin for Hermes that fades unused memories — RileyRalmuto · 2026-11-21
- Study: evolutionary LLM search found valid solutions but picked invalid programs in 2 of 3 runs — Unusual-Spot-5846 · 2026-11-21
- Dev routes tool calls through a 421M model at 12ms to catch risky agent actions — looselyhuman · 2026-11-21
- Screen-share + voice via MCP into Claude: a faster UI iteration workflow — Different_Scholar_74 · 2026-11-21
- Tencent & KCL release GroupAssistBench: group-chat agents recall memories but fail to use them — 量子位 · 2026-11-21
- Open-source E2E-encrypted MCP server lets your phone message a local AI agent — uriwa · 2026-11-21