SpectraReward: Zero-Shot Image Reward Model

ByteDance-Seed · hf · 2026-07-15

ByteDance Seed published a paper on SpectraReward, proposing a training-free, preference-annotation-free reward function that uses pre-trained MLLMs as reward models for text-to-image generation.

Core Concept

Instead of directly asking the MLLM to score the generated image, it checks whether the original prompt can be recovered from the generated image.
The reward is calculated as the prompt's average log-likelihood via a single image-conditioned, teacher-forced forward pass.
For unified multimodal models, it also proposes Self-SpectraReward: the model's "understanding branch" acts as the reward model for its "generation branch," forming a closed-loop self-improvement cycle.

Experimental Results

Covers 2 diffusion models, 3 RL algorithms, 9 reward MLLM backbones, 4 MLLM families, 4B to 235B parameters, and 5 OOD text-to-image benchmarks.
Results show that both SpectraReward and Self-SpectraReward consistently improve generation performance, outperforming previous MLLM-derived reward training methods.
The paper also notes that larger reward MLLMs are not necessarily better; Self-SpectraReward can even match or exceed larger external reward models, indicating that reward-policy alignment is crucial.

Project Page: https://huangrh99.github.io/SpectraReward/

Original post →

SpectraReward: Zero-Shot Image Reward Model

More from Multimodal