ByteDance's Seedream Team Proposes VoT: Visual Thinking Before Rendering

JingxiangSun42 · x · 2026-09-13

ByteDance's Seedream team released the paper VoT: Vision-of-Thought for Unified Multimodal Representation Alignment.

Current text-to-image systems use a "text encoder + diffusion decoder" paradigm where text semantics directly modulate continuous latent noise, lacking an explicit, interpretable intermediate representation bridging high-level semantics and low-level visual signals.
VoT inserts a discrete visual-thinking layer between VLMs and DiTs: instead of treating VLMs as mere text encoders, they act as multimodal planners that generate discrete VoT tokens encoding high-level visual plans (objects, layouts) before rendering pixels.
A specialized VoT tokenizer is trained with a closed-loop objective combining VLM alignment, feature reconstruction, and vector-quantization losses, making tokens semantically readable by the VLM while preserving visual information for generation.
Experiments show VoT improves semantic alignment and provides a structured interface for interpretable, controllable generation.

More from Multimodal