Image Tokenizers Define the Visual Language of Unified Multimodal Models

peterxichen · x · 2026-09-15

In a 9-post thread, Siting Li examines image tokenizers through the lens of unified autoregressive multimodal training.
Core argument: tokenizers do more than compress and reconstruct pixels — they define the visual language a single model must learn, align with text, and use for both understanding and generation.
Counterintuitive takeaway: the best compressor may not produce the easiest language for a model to learn, so compression quality and learnability should be evaluated separately.
Tokenizer choice directly affects training outcomes for unified understanding+generation models, making it a first-class design decision.

More from Multimodal