ByteDance Releases EdgeBench to Measure Agent Learning in Real-World Environments

字节跳动Seed · wechat · 2026-07-07

ByteDance's Seed team released EdgeBench, an ultra-long-horizon evaluation set designed to measure an Agent's ability to continuously learn and improve in real-world environments. The benchmark includes 134 diverse tasks spanning six domains: science, complex software engineering, white-collar knowledge work, algorithm optimization, frontier mathematics, and digital games. Each task allows an Agent to work continuously for at least 12 hours (with some extended experiments exceeding 72 hours). Curated by domain experts tackling real problems, over 90% of the tasks are newly built. Human experts take an average of 57.2 hours to complete a single task, with the maximum reaching 320 hours.

Based on roughly 38,000 hours of environment interaction logs, the team found that an Agent's overall performance in environmental learning follows a highly precise log-sigmoid curve, with an average fit accuracy of R²=0.998, which can be explained from a graph exploration theory perspective. Looking at frontier models across different generations, Agent learning speeds roughly double every three months. Learning trajectories vary across tasks—some show steady improvement, some plateau after rapid early progress, and others experience breakthroughs only after long periods of stagnation.

Currently, 51 of these tasks and the complete evaluation framework have been open-sourced for the community to study how Agents learn from real environments. The project, paper, code, and data are all publicly available.

Original post →

ByteDance Releases EdgeBench to Measure Agent Learning in Real-World Environments

More from coding & agent