RExBench: best coding agent solves only ~33% of autonomous AI research extension tasks
najoungkim · x · 2026-10-16
RExBench (ACL 2026) benchmarks whether LLM agents can autonomously implement extensions to 12 real research papers with expert-written instructions and executable success criteria. Across 12 agents built with aider and OpenHands, none solved a majority of tasks: the best reached 33%, and under 44% even with human hints—showing current agents remain far from autonomous research extension.
More from coding & agent
- Developer rebuilds GitHub in 4 days: Keel makes agents declare intent before changing code — threepointone · 2026-10-16
- Stop using small models as judges with overloaded context, dev warns — xeophon · 2026-10-16
- Meta's open-weight Muse Spark Open hits Ollama with 1M-token context and Claude Code support — ollama · 2026-10-16
- Monid launches: one balance, pay-per-call access to 1,700+ tools for AI agents — HeyAmit_ · 2026-10-16
- OpenAI publishes Sites tutorial: build and share your first Site — OpenAIDevs · 2026-10-16
- OpenAI Sites goes GA for Enterprise with US/EU data residency — OpenAIDevs · 2026-10-16