One-shot RL repro: training Qwen3.5-4B to paint watercolour flowers with a judge model
charles_irl · x · 2026-10-22
A Modal engineer one-shot reproduced the community RL experiment that taught a model to paint watercolour flowers, and published a full tutorial. Charles quipped "let a thousand policies contend."
Key details:
- Model writes p5.brush code to produce watercolour flower sketches, inspired by a community blog post
- Trains Qwen3.5-4B; the reward comes from Qwen3.6-27B acting as a judge doing pairwise comparisons against the HuggingEnvs/watercolour-reference-pool
- Sketches are rendered to PNGs in Modal Sandboxes each rollout before judging
- Full code, reward curves, and sample outputs included; reproducible on Modal Dojo
More from coding & agent
- OpenLore gives every coding agent the same memory with one folder of Markdown — Arindam_1729 · 2026-10-22
- Walking bento box built with Claude Code + Blender Python, no MCP needed — sidahuj · 2026-10-22
- Claude Code Autonomously Generated 2,000+ Krea2 Images Overnight — reynadsaltynuts · 2026-10-22
- Indie dev building game #4 with AI starts with a central game asset management hub — ezshine · 2026-10-22
- Diffy: a cross-platform desktop code review app with a custom semantic diff engine — seatedro · 2026-10-22
- Open-Source Garmin MCP Connector Plugs Workout Data Into AI Agents — Vast-Grapefruits · 2026-10-22