Verifiable Visual Rewards lift SD3.5 instruction accuracy from 2.8% to 28.3% on arXiv
testingcatalog · x · 2026-09-29
Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov, and Luke Zettlemoyer released the arXiv paper "Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts."
- Problem: precise instruction following (object counts, spatial relations) in image generation remains open, partly because training relies on unreliable reward models (object detectors, VLMs)
- Method: VVR is the first programmatically verifiable image reward framework — each task is a geometric scene from which prompt and deterministic verifier are auto-generated at any scale and complexity
- Data: releases VVRBench (10,000 tasks, 32 constraint types) and VVRBench-Challenge (720 harder tasks); strongest evaluated model GPT-Image-2.5 solves just 21.4%
- Results: RLVVR raises Stable Diffusion 3.5 Medium's VVRBench accuracy from 2.8% to 28.3% with consistent easy-to-hard and out-of-domain generalization; mixing VVR into existing objectives further improves quality and human preference, motivating adoption in standard post-training recipes
33-page paper with code and benchmark open-sourced.
More from Multimodal
- Testing Qwen Image 2.1 and Minimax H3 on anime-to-photo conversion and film photo restoration — Ant_6431 · 2026-11-16
- Second take of a 'Cubee run' video generated with Minimax H3 — JahJedi · 2026-11-16
- Non-artist builds 14-chapter hand-painted storybook game with Fable 5.2 in 4 hours — jradoff · 2026-11-16
- Subject LoRAs on Minimax H3 Image learn poor likenesses while identical datasets excel on Qwen 2.1 — Ad_Organic · 2026-11-16
- Seedance-generated anime train scene stuns Reddit with cinematic quality — Ok-Solution-7889 · 2026-11-16
- AI-generated 'Men in Black' reimagined in 1977 goes viral — techhalla · 2026-11-16