Qwen3.5 plays Pokémon by reading the screen while a compiled network runs the Game Boy CPU

vincent_koc · x · 2026-11-02

vincentkoc shares Qwengameboy, an engineering-notes project where AI ran the whole experiment end-to-end, including write-up and interactive visuals.

The trick: unmodified Qwen3.5 reads a text version of the screen (a grid of digits) and picks a button. Next to it sits a neural network that a compiler built rather than trained — it executes the Game Boy CPU one instruction at a time. A small Python program connects the two and paints the screen.

Verification: the compiled CPU ran 224 million instructions from the cartridge's startup code, with the final frame matching a reference emulator pixel for pixel; another run from a save state produced 117 consecutive frames matching exactly. The LLM only handles a simpler test loop — reading scenes and choosing buttons — while AI also wrote up the results.

Original post →

More from Fun

Fun channel →