Kaggle and Eigent launch AgenticRL-Bench to test coding agents on real RL incidents
dejavucoder · x · 2026-11-20
Kaggle, in collaboration with EigentAI, announced AgenticRL-Bench, a benchmark testing whether coding agents can diagnose and fix real incidents across the agentic RL stack (Miles, slime, verl, SGLang, Megatron, Harbor).
- Built from 22 post-training RL tasks; the Kaggle leaderboard scores agents on the 19 verifiable ones.
- Tasks mirror real agentic-RL engineering work: Verl Upgrade Gate, Reward Determinism under Load, Trial Fault Attribution, SQL Partial Credit Exploits.
- A task passes only if every hidden check passes; each agent gets three attempts per task, run by Kaggle on Modal.
More from coding & agent
- Self-hacking with abliterated models: biggest risks were infra side channels, not app code — cyb3rops · 2026-11-20
- Pocket City: a pixel-art Amsterdam canal simulation built entirely on real geodata — anselm · 2026-11-20
- Mnemos vs Claude Code memory test details: 65%/41%/42% recall across three blind tests — RileyRalmuto · 2026-11-20
- Solo-built Mnemos memory system recalls 2x more facts than Claude Code's native memory — RileyRalmuto · 2026-11-20
- Suno MCP now lets you generate original songs inside ChatGPT and Claude — heyshrutimishra · 2026-11-20
- OpenAI Shows How to Connect Team Data and Custom MCP Servers to ChatGPT — OpenAI · 2026-11-20