Dev turns Pokelike.xyz into an open-source LLM and RL benchmark harness
Logical_Delivery8331 · reddit · 2026-08-19
A data scientist built a full harness around the web game Pokelike.xyz, turning it into a dual LLM and reinforcement learning benchmark, now open source.
LLM side: the model receives a system prompt, tools, previous turns, and the current game state per turn — all configurable (strategy prompt, state representation, toolset). Starter bots are included in the repo.
Early results: GLM 5.2 at best reaches the second map before dying; Opus performs about the same. The author wants to isolate whether the bottleneck is strategy, missing information, state representation, or tooling — and hopes someone gets better results from smaller models via prompt/tool improvements.
Traditional RL agents run alongside on a shared leaderboard. Everything runs fully offline (game and assets downloaded once, simulated locally). Built by the author, a friend, and Claude Code.
Repo: https://github.com/pierpierpy/pokelike.xyz.bot
More from coding & agent
- Sub8 enables local Linux automation controlled by Claude or local models — Daniel_Farinax · 2026-08-19
- Always-on AI Agents need background updates without restarts, hitting OS-level limits — koltregaskes · 2026-08-19
- Open Source MemMachine: Universal Memory Layer for AI Agents — tom_doerr · 2026-08-19
- Rust Web Framework Earendil Hits Hacker News Front Page — mitsuhiko · 2026-08-19
- Fix Hallucinations with Local Wikipedia, Not Bigger Models — PieBru · 2026-08-19
- Yeku: An MCP Server to Deploy Full Apps in One Message — itsAg3nt47 · 2026-08-19