Agent Plasticity paper proposes benchmarking how efficiently AI agents learn from experience
anirudhg9119 · x · 2026-10-09
- A new paper (arXiv 2610.08902) by Harman Singh, Jason Weston, Anuj Mahajan et al. introduces "agent plasticity": how efficiently an agent converts experience into gains in future held-out performance, rather than measuring static capability.
- Method: agents amortize past experience into reusable artifacts inherited by later instances; performance is measured at checkpoints on training and held-out environments, accounting for learning cost.
- Findings: frontier models show sharply divergent improvement trajectories despite comparable learning opportunities; some gain persistent improvements, others stay near baseline, and in-distribution gains often only partially transfer out-of-distribution. Oriol Vinyals endorsed it as a good ICL benchmark.
More from coding & agent
- A Playable 3D Boat Game Embedded in an X Post, Built by Claude Opus — prasenx · 2026-10-09
- edith-1 monitors agent traces, beats Sonnet-5.5 on balanced accuracy at 1/274 the cost — xennygrimmato_ · 2026-10-09
- How edith-1 works: probabilistic filter plus expensive agent judge for flagged runs — xennygrimmato_ · 2026-10-09
- Solari launches agent infrastructure: 8ms browsers, 10x faster than Browserbase — Scobleizer · 2026-10-09
- Splash 1.3.0 cuts local agent first-token latency from 19s to 1s via SSD offloading on M6 Mac — BeidiChen · 2026-10-09
- OpenAI makes coding-agent steering instant, ships new "GPT-6.1 Sol ultrafast" release — Dimillian · 2026-10-09