World Model Training Without Action Labels
bronzeagepapi · x · 2026-07-16
This shared post discusses a latent world model training method (NEO) that does not require action labels.
Core Idea
- Unlike methods such as DINO-WM or LeWorldModel that optimize action sequences using CEM or gradient descent during inference, NEO learns an action codebook via VQ-VAE.
- During inference, the model can directly select and execute actions from the codebook.
- A key feature is that no action labels are needed; training can begin directly given a video.
Author's Concerns
- Training can be highly unstable if the data involves massive dynamic shifts or heavy noise.
- The architecture itself is already quite complex. The author prefers simpler methods like VICReg, SIGReg, or VISReg for easier experimentation with different applications (e.g., LeWorldModel).
- Action search is still necessary if the current action sequence fails to reach the target state.
Overall, it's a highly interesting world model approach, albeit with stability and complexity issues.
More from Research
- Style-similarity analysis puts Kimi K3 closer to Claude Fable 5 than to K2.6 — soumitrashukla9 · 2026-07-21
- A GLP1R variant may explain stronger Ozempic weight loss, and the team built an agent workflow — julia_kiseleva · 2026-07-21
- Proceedings for the second geometry-grounded representation learning workshop are now online — erikjbekkers · 2026-07-21
- New survey maps how agentic systems are learning to improve themselves — SchmidhuberAI · 2026-07-21
- A curated TTS list for voice agents tracks latency, cancellation, and evals — mahimairaja · 2026-07-21
- Jacob Tsimerman interview frames LLMs as a turning point for mathematical discovery — stevenstrogatz · 2026-07-21