Baidu ERNIE team unveils HiUni: one dense backbone unifies visual understanding and generation

ernie-research · hf · 2026-11-20

Baidu's ERNIE team presents HiUni, extending a pretrained VLM into a unified multimodal model with heavy parameter sharing: the same language-aligned ViT and dense Transformer backbone handles both understanding and generation, with task-specific computation confined to the output side. A Pixel Diffusion Head tackles pixel-space optimization, and a lightweight HiGate pathway restores fine spatial details lost to ViT abstraction. HiUni matches the base VLM on understanding while staying competitive on image generation and editing.

Original post →

More from Multimodal

Multimodal channel →