ByteDance's Seedream Team Proposes VoT: Visual Thinking Before Rendering
JingxiangSun42 · x · 2026-09-13
ByteDance's Seedream team released the paper VoT: Vision-of-Thought for Unified Multimodal Representation Alignment.
- Current text-to-image systems use a "text encoder + diffusion decoder" paradigm where text semantics directly modulate continuous latent noise, lacking an explicit, interpretable intermediate representation bridging high-level semantics and low-level visual signals.
- VoT inserts a discrete visual-thinking layer between VLMs and DiTs: instead of treating VLMs as mere text encoders, they act as multimodal planners that generate discrete VoT tokens encoding high-level visual plans (objects, layouts) before rendering pixels.
- A specialized VoT tokenizer is trained with a closed-loop objective combining VLM alignment, feature reconstruction, and vector-quantization losses, making tokens semantically readable by the VLM while preserving visual information for generation.
- Experiments show VoT improves semantic alignment and provides a structured interface for interpretable, controllable generation.
Related event: ByteDance Seedream Team Introduces VoT: Thinking Before Rendering(3 posts)→
More from Multimodal
- NVIDIA open-sources NemotronLabs RE-USE, a 9.6M-param multilingual speech denoiser — NVIDIAAI · 2026-10-31
- AI-generated fake trailer imagines a 1999 Dino Crisis movie with retro VHS feel — noizlab_studio · 2026-10-31
- Spatial Hub aggregates ~5000 public COLMAP-compatible spatial datasets for world models — janusch_patas · 2026-10-31
- Single image to walkable 3D Gaussian splat scene using World Labs Atlas and WebGPU — XRarchitect · 2026-10-31
- AI video reimagines Suicide Squad's Joker as a rapper in a full music-video treatment — Silver_War5476 · 2026-10-31
- 60+ community-made ChatGPT slash-style words turn one word into a full visual style — heyshrutimishra · 2026-10-31