AgentOdyssey, a Text-Game Engine for Test-Time Continual Learning Agents, Accepted at NeurIPS
DanielKhashabi · x · 2026-09-26
The Johns Hopkins team behind AgentOdyssey announced NeurIPS acceptance. AgentOdyssey is an open-ended long-horizon text game generation engine for evaluating test-time continual learning agents.
- Core idea: real agents can't separate training from testing and must learn continuously from interaction; the framework breaks the classic "no learning at test time" assumption.
- Five abilities evaluated: exploration, episodic memory, world knowledge acquisition, skill learning, long-horizon planning.
- Method: procedurally generates open-ended text games; diagnostics cover progress, knowledge acquisition, memory, action diversity, and model cost.
- Findings: even top agents remain far below human performance; short-term memory benefits multiple agent paradigms and is key to agentic test-time training.
Open source: pip install agentodyssey.
More from coding & agent
- Peter Steiberger says he codes with Codex plus an OC harness, not Claude — steipete · 2026-09-26
- One prompt makes Claude generate a 2-min video game history film entirely in code — prasenx · 2026-09-26
- Run /doctor prompt-audit to find stale paths and conflicting instructions in Claude Code — daniel_mac8 · 2026-09-26
- Repurpose your old M1/M2 Mac as a local test server with this open-source runner script — jasonkneen · 2026-09-26
- DHH says 37signals went "pencils down": 150k lines in August with agents vs 30k/year by hand — Recent-Tangerine2745 · 2026-09-26
- Harness engineering: the key discipline for building reliable AI agents — techNmak · 2026-09-26