Agentick benchmark accepted at NeurIPS: LLM vs RL agents on same tasks, no single winner

pcastr · x · 2026-09-28

Agentick, a unified benchmark for training and evaluating general sequential decision-making agents, has been accepted to NeurIPS E&D.

RL agents, zero-shot LLMs, VLMs, hybrids, bots, and humans are all evaluated on the same tasks, seeds, and score.
The goal is to put zero-shot LLM agents and neural nets trained from scratch with RL on common ground to study their respective strengths and limitations.
First result: no single agent dominates.
Paper, code, and blog are available.

More from coding & agent