W&B says NVIDIA’s 14B DreamZero model fails hard tasks despite clean scalar metrics
wandb · x · 2026-07-23
- W&B reports zero-shot evaluation of DreamZero, NVIDIA GEAR Lab’s 14B World Action Model, across 65 rollouts.
- On the hardest task, scalar metrics failed to separate success from failure, showing that aggregate numbers can miss important behavior.
- The full eval is used to inspect where the model generalizes and where it breaks.
More from Embodied
- FLASH: GPU-native deformable sim learns cloth folding in minutes on one RTX 5090 — ChongZzZhang · 2026-10-26
- Luke Wroblewski spots it: iPhone Duo's corner radius mirrors the original iPhone — LukeW · 2026-10-26
- Humanoids shattered forecasts: half-marathon record in 2026, 100m faster than Bolt — ben_j_todd · 2026-10-26
- Humanoid walking is continuous correction — watch recovery, not locomotion, in demos — Olivier__OG · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26