Training a 9B model with GRPO to build low-poly Blender rooms: lessons learned

TheMoonMidas · x · 2026-09-08

A hands-on RL experiment trains a small policy model to write Blender Python code, building complete low-poly isometric rooms from blank geometry via OpenEnv.

Setup: Qwen3.5-9B policy over studio/bedroom/kitchen/living-room prompts (256 train, 64 eval), judged by GLM-5.3-Flash through HF Inference Providers; baseline of 16 rollouts before a planned 200 GRPO steps.

Key lessons:

Align geometry with deterministic rewards plus VLM-as-a-judge;
GEPA with various rewards only got the model to valid rooms — nothing more;
Adding detail after geometry is solved is very hard;
Stricter reward functions yield less advantage (task too hard), and GRPO didn't help;
Image guidance to both policy and judge didn't improve results; "flowers can be abstract, a house can't be."

The environment, scripts, and run docs are open-sourced on Hugging Face (merve/blender-grpo-gepa).

Original post →

Training a 9B model with GRPO to build low-poly Blender rooms: lessons learned

More from coding & agent