A Minute of 30fps Video Is 7M Tokens: Why Video LMs Still Need Visual Encoders
HildeKuehne · x · 2026-10-19
A discussion questioning video tokenizer designs that skip an image encoder: a 2K image is 4K tokens, a minute of 1fps video is 200K tokens, and at 30fps that's 7M tokens—while you can chat with an LLM for hours before hitting 100K tokens. Visual patches are far less informative on average than language tokens, so a visual encoder is still needed to pool spatiotemporally or selectively drop patch tokens, preventing redundant information from flooding the LLM.
More from Multimodal
- SenseTime co-founder: encoder-free native VLMs key step toward unified multimodal AI — liuziwei7 · 2026-10-19
- Reddit user shares "Brutus Part 1", a dark fantasy short film — akasan · 2026-10-19
- Krea2 vs MOSS vs Qwen2.1 vs Flux3 hands-on: Flux3 flops hard — CaptainBackwoods447 · 2026-10-19
- Animator uses image models as fast style tests, concludes hand-drawing wins — lmoroney · 2026-10-19
- AVC-Reward: an event-centric framework for evaluating audio-video generation consistency — ZhejiangUniversity · 2026-10-19
- Tsinghua and Z.AI study shows image-generation training expands unified multimodal models' knowledge — zai-org · 2026-10-19