ByteDance EdgeBench: Evaluating AI Agents' 10-Hour Self-Iteration Capabilities
karminski3 · x · 2026-07-06
ByteDance's Seed team released the EdgeBench evaluation framework, designed specifically to assess AI agents' ability to autonomously iterate and improve over long horizons (up to dozens of hours) in a real, runnable environment. Unlike single-task evaluations, it focuses on whether an agent can leverage environmental feedback, adjust strategies based on failures, and continuously enhance performance.
The tester used a locally run 35B-A3B model (4-bit quantization) to tackle the classic Roguelike game DCSS (command-line version). The task required the Agent to write Lua bot scripts via CodeX CLI to automatically explore, fight, and descend levels within roughly 70 minutes. The initial automated score was 15.2, which improved to a maximum of 33.4 after multiple iterations, clearly showcasing the AI's true learning trajectory in long-horizon tasks.
Related event: ByteDance Releases EdgeBench to Evaluate Long-Horizon Agent Evolution(4 posts)→
More from coding & agent
- OpenAI ships GPT-6.1 Terra; Brockman says its computer use now beats his own speed — gdb · 2026-10-27
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- Dev argues Claude burns far more tokens than GPT models for the same coding work — haider1 · 2026-10-27
- Stop building agents that can't change models: stable contracts, external state, real evals — rseroter · 2026-10-27
- Cheap models can cost more in agent harnesses than frontier ones, researcher warns — vasuman · 2026-10-27
- Kanzo: A 5k-Line Rust Kanban Binary With MCP for Claude Code and Codex — allmudi · 2026-10-27