Tencent Youtu releases Youtu-Parsing-Omni, a one-encoder all-modality parser
tencent · hf · 2026-10-30
Tencent Youtu released Youtu-Parsing-Omni on Hugging Face, a unified multi-modal parser that encodes documents, images, audio, and video into structured spatiotemporal records using shared weights and cross-modal fusion.
- Architecture: vision and audio stems form a single input sequence passed through shared Transformer blocks with fusion layers, decoded by a Youtu-LLM decoder into a unified OmniSchema JSON.
- Coverage: documents (Markdown/LaTeX/OTSL tables), natural images with grounded objects, charts (Mermaid/tables), geometry, chemical structures (SMILES), sheet music (ABC notation), speech/sound with timed ASR, and audio-visual video timelines.
- Training: a self-evolving on-policy distillation procedure, claimed to achieve state-of-the-art results across parsing benchmarks.
More from Multimodal
- NVIDIA's multilingual speech dataset Granary-v2 trends on Hugging Face — nvidia · 2026-10-30
- Oxford's Holo4D uses holistic 4D reconstruction as geometric control for video diffusion — Oxford · 2026-10-30
- 100% AI-made short film reimagines ancient Egypt with BMX bikes, full 5-step workflow shared — socialwithaayan · 2026-10-30
- 90 open-source apps in two weeks: the full #OpenSourceHiggsfield community catalogue — Visual_Tiger_2400 · 2026-10-30
- OpenSuperComputer open-sources a Higgsfield-style automated content generation pipeline — iAM_A_NiceGuy · 2026-10-30
- Open SuperComputer: open-source chat-driven workspace for ComfyUI media workflows — iAM_A_NiceGuy · 2026-10-30