ByteDance Releases EdgeBench to Measure Agent Learning in Real-World Environments
字节跳动Seed · wechat · 2026-07-07
ByteDance's Seed team released EdgeBench, an ultra-long-horizon evaluation set designed to measure an Agent's ability to continuously learn and improve in real-world environments. The benchmark includes 134 diverse tasks spanning six domains: science, complex software engineering, white-collar knowledge work, algorithm optimization, frontier mathematics, and digital games. Each task allows an Agent to work continuously for at least 12 hours (with some extended experiments exceeding 72 hours). Curated by domain experts tackling real problems, over 90% of the tasks are newly built. Human experts take an average of 57.2 hours to complete a single task, with the maximum reaching 320 hours.
Based on roughly 38,000 hours of environment interaction logs, the team found that an Agent's overall performance in environmental learning follows a highly precise log-sigmoid curve, with an average fit accuracy of R²=0.998, which can be explained from a graph exploration theory perspective. Looking at frontier models across different generations, Agent learning speeds roughly double every three months. Learning trajectories vary across tasks—some show steady improvement, some plateau after rapid early progress, and others experience breakthroughs only after long periods of stagnation.
Currently, 51 of these tasks and the complete evaluation framework have been open-sourced for the community to study how Agents learn from real environments. The project, paper, code, and data are all publicly available.
Related event: ByteDance Releases EdgeBench to Evaluate Long-Horizon Agent Evolution(4 posts)→
More from coding & agent
- AIE Code Summit 2026 opens applications: 800 seats, keynotes streamed free — trq212 · 2026-10-27
- TRL adds Score Centering, a simple fix for RLHF training instability — lvwerra · 2026-10-27
- Precompute golden answers and match at runtime for exact wording control — colinmcnamara · 2026-10-27
- Dev argues there's still alpha in working with your coding agent interactively — alex_frantic · 2026-10-27
- Researchers launch AMA on patterns for safer AI agents used by citizen developers — Loose_Rip359 · 2026-10-27
- Gradio Adds In-Chat Human-in-the-Loop So ML Intern Pauses Before Training — Gradio · 2026-10-27