ScholarEvolve: Agents That Read Papers to Evolve, +14pp on AppWorld
机器之心 · wechat · 2026-10-21
Researchers from UCSB and Microsoft propose ScholarEvolve, an approach where an agent improves its own harness by reading academic literature like a human researcher would — no model weight changes required.
Method
- Decomposes the harness into five modules (tools, context, skills, memory, workflow) for targeted improvement
- A Research Agent analyzes execution traces, retrieves relevant papers from arXiv, and uses topic modeling to cover diverse technical approaches rather than similar ones
- A Coding Agent implements paper methods (including GitHub code and appendix prompts), then combines and evaluates modules on validation tasks
Results
- Qwen3.5-27B on AppWorld Challenge: 49.6% → 63.6% (+14pp), vs 54.6% for MetaHarness; AppWorld Normal: 69.0% → 81.4%
- GPT-5.4-mini: AppWorld Challenge 46.1% → 55.6%; τ²-Bench Telecom 72.7% → 81.9%
- Ablations confirm literature guidance, modular search, and topic modeling all matter; individually best modules don't always combine into the best harness
Lifelong evolution: simulating three consecutive paper-publication windows, performance climbed from 69.0% to 81.5% as new papers arrived — suggesting agents can continuously absorb research advances.
Related event: Microsoft's ScholarEvolve Lets Agents Self-Improve by Reading Papers(2 posts)→
More from coding & agent
- Google and Meta bring native TPU training to TorchTitan, supporting Llama 3 and Qwen 3 pretraining — rseroter · 2026-10-22
- Tsinghua's Design Anything uses intent-execution graphs to keep design agents on-brief at scale — Xuchao Ren · 2026-10-22
- Open source is dead in the coding-agent era: power shifts from skill to capital — MarcJSchmidt · 2026-10-22
- Manim-style presentation template built with Claude and Codex, now open-sourced — mmbronstein · 2026-10-22
- Dev builds Mac browser with MCP server so Claude Code can browse with real logins — DhammaIY · 2026-10-22
- Maggie Appleton's new talk: better boundary objects and thicker interfaces for planning with agents — IanArawjo · 2026-10-22