NVIDIA Open-Sources PixelUMM: Encoder-Free Model Handling Image/Video in Raw Pixel Space
udmrzn · x · 2026-10-05
NVIDIA researchers open-source PixelUMM, an encoder-free unified multimodal model that works directly in raw pixel space for both image and video understanding and generation — no VAE, no ViT. Built on Qwen3-8B as a Mixture-of-Transformers with shared self-attention over text, clean and noisy pixels; images become 16×16 patches, videos 4-frame tubes. Performance matches open-source baselines. Paper, code and weights are fully available.
Related event: NVIDIA Open-Sources PixelUMM, a 15B Encoder-Free Unified Multimodal Model(2 posts)→
More from Research
- RFdiffusion 4 generates putative binder for blood-brain-barrier receptor CD98hc with ipTM 0.88 — AllThingsApx · 2026-10-25
- DeepMind-backed benchmark: best AI agent solves 49/50 unpublished research-level physics problems — rohanpaul_ai · 2026-10-25
- Researchers rebuild city power grid topology from open data, no utility secrets needed — anselm · 2026-10-25
- EMNLP 2026 paper exposes hidden representational instability in VLMs — PMinervini · 2026-10-25
- AI fixes 95.8% of failed spinal-cord labels: nnU-Net rescues patient MRI data — bravo_abad · 2026-10-25
- MemOS teases memt1.0: memory-native MoE architecture, ~5% coding gains — CuriousSiopao · 2026-10-25