Thesis project trains LLMs to paint with code via RL, making images editable like programs

thomasahle · x · 2026-09-27

A thesis project by Surya Narreddi and Cameron Franz uses reinforcement learning to train a language model that generates images as editable p5.brush JavaScript sketches — code is the artifact, so you can edit the output granularly instead of re-prompting the model.

How it works (four-step loop, run thousands of times)

The model gets a prompt (e.g. "draw a peach hibiscus in watercolour") and writes a complete JS sketch;
The sketch is rendered to PNG in a sandboxed Puppeteer environment;
Two random reference paintings from a hand-rated pool are sampled, and a separate judge model picks the better watercolour;
The judgment becomes a reward signal, GRPO updates the model, and the loop repeats.

The deeper research question is how to run RL on creative/design tasks: rewards must be verifiable, but aesthetics are neither right nor wrong. The design problem becomes the reward function and the judge's criteria — too rigid and the model converges, too loose and it drifts.

Original post →

Thesis project trains LLMs to paint with code via RL, making images editable like programs

More from Multimodal