The Resolution Hypothesis: agent evolution stalls because noisy evals can't spot real improvements
ShanghaiTech · hf · 2026-11-19
A ShanghaiTech + ByteDance + BIGAI + SUSTech paper, "The Resolution Hypothesis," argues that agent evolution (proposers generating prompts/skills/memories, selection keeping higher scorers) stalls not because methods fail to propose better artifacts, but because noisy evaluations can't reliably identify them.
Across 7 evolution methods, 6 models and over 10M LLM calls, the hypothesis holds at two levels:
- Candidate level: helpful artifacts exist, but selection is no better than random at telling them from harmful ones, so errors accumulate across rounds.
- Method level: methods trade wins across papers — exactly what noisy evals would produce.
Implication: progress requires making true improvements detectable and accumulable, rather than piling on new proposal methods.
More from coding & agent
- Microsoft ships Copilot SDK 2.0 with Rust core, powering all Copilot agents — PaulShellDev · 2026-11-19
- Rediscovering Notion as a Full Context Front End for AI Workflows — ivanhzhao · 2026-11-19
- WebMCP polyfill lands on npm: let web pages expose tools to browser agents — philnash · 2026-11-19
- Ex-DeepSeek core member open-sources Linger, a local AI agent claiming 5x faster inference — Aiden_Tech_Ai · 2026-11-19
- Hand-scoring my AI action checker: most checks were wrong more often than right — chich2017 · 2026-11-19
- Meta's Pyrefly 1.5 hits 100% typing spec conformance, checks 40% faster — tdhopper · 2026-11-19