Contrastive World Models: dropping pixel reconstruction beats Dreamer in visually complex environments
bonniesjli · x · 2026-09-26
Bonnie Li released an arXiv paper on Contrastive World Models, a way to learn latent dynamics without pixel reconstruction:
- Method: Built on Dreamer, it replaces observation reconstruction with a Deep InfoMax-style lower bound that maximizes mutual information between state-action sequences and local patch features of future observations, so representations keep only planning-relevant information.
- Results: In small-scale experiments, it matches Dreamer and a momentum-prediction baseline in default settings, and substantially outperforms both once distractors or natural video backgrounds are introduced, while training faster since the pixel decoder is removed entirely.
- Takeaway: Written independently between roles with minimal compute; infomax-based objectives look like a principled path to world models robust to visual nuisance factors—key for transferring model-based RL agents to the real world.
Related event: Contrastive World Models: Pure Latent-Space Training Beats Dreamer(5 posts)→
More from Embodied
- Luna, a jewelry-style AI wearable that speaks before you ask, launches on Kickstarter — nikola_mr64990 · 2026-11-11
- Dev reverse-engineers Anker AI recording earbuds, ships open-source client app — vista8 · 2026-11-11
- Simulated hand with 39 muscles learns 9/9 poses after 426M training steps — MaziyarPanahi · 2026-11-11
- Versor pitches real2sim2real: robotics sim needs physical dynamics, not just visuals — abhishekunique7 · 2026-11-11
- Origami Robotics and PalatialSim unveil real2sim2real Dexterity Factory, 100 open datasets next week — ybisk · 2026-11-11
- Trainable JWST model boosts exoplanet sensitivity 2–15x in Nature paper — bravo_abad · 2026-11-11