Sooool: latency-aware agent plays Touhou in real time, repo and guide open-sourced
wzenus · x · 2026-10-17
wzenus open-sourced Sooool, where the model plays real-time games frame-by-frame as a visual policy — no code-as-a-policy. The core challenge: the world keeps moving while the model thinks, so actions may be stale on arrival. Their 'autoresearch' loop:
- Runs up to 8 model requests in parallel while earlier decisions are in flight
- Simulates target API latency with cheaper inference during development
- Provides latency mean/std/P90/P95 so the agent can plan for where things will be when its action lands
- Runs episodes, inspects failures, and updates instructions each round
Cost is about $50 per task in paid API calls, mostly final verification. Learned lessons include 'predict where the boss will be, not where she was' and 'more damage isn't enough if it costs more hits.' Repo includes experiment logs and a reproduction guide.
Related event: Sooool Open-Sourced: Models Play Real-Time Games Frame by Frame(2 posts)→
More from coding & agent
- Sol plays Asterix from screenshots, holding each move until its next reply arrives — wzenus · 2026-10-17
- GPT-6 Sol plays rhythm games in real time, picking keys and tap times from frames — wzenus · 2026-10-17
- Sol picks speed and lane targets frame-by-frame from onboard camera while racing — wzenus · 2026-10-17
- Teknium catalogs 330 hackable devices and 1,697 project ideas for Hermes Agent — Teknium · 2026-10-17
- Garry Tan launches GBrain, a shared team memory synced to ChatGPT, Claude and Cursor — garrytan · 2026-10-17
- All technical work is becoming evals: engineers' job is building agent verification loops — rachittshah · 2026-10-17