AgenticRL-Bench debuts: best agents pass only 14% of real-world agentic RL tasks
cuijiaxun · x · 2026-11-12
Eigent AI's Environment Lab released AgenticRL-Bench, the first benchmark where agents perform agentic RL work themselves—22 tasks rebuilt from real incidents in production agentic RL systems (Miles, Slime, Verl, SGLang, Harbor) across 8 stack areas.
Key results:
- 6 agents × 3 attempts: on the 19 verifiable tasks, no agent exceeds 14.0% pass@1 (Claude Opus 5.5 leads; the weakest scores just 1.8%)
- On 3 open-ended tasks, agents improved real policies/training steps using 4×H200 GPUs over 10–16 hours; in one task where the base model fails all 38 items, the best attempt developed its own method
Takeaway: top agents are far from autonomously running agentic RL research, but early signs of autonomous method discovery are emerging.
More from coding & agent
- Anthropic details agent memory: dreaming organizes Claude's memory, wiki+search hits 61% — RLanceMartin · 2026-11-12
- Opinion: AI infra is becoming a coding runtime as intelligence units outgrow tokens — prateekj · 2026-11-12
- Claude Docs/Slides block external MCP clients: no public API, export only — steipete · 2026-11-12
- Anthropic explains agent memory: 'dreaming' lets Claude consolidate session transcripts into notes — rseroter · 2026-11-12
- Three AI game dev competitions land this month, one with a $47,800 prize pool — Promptmethus · 2026-11-12
- PrefixRelay: open-source proxy gives Cursor 97%+ KV cache hits on local models — jjusko20 · 2026-11-12