Sekai2: A 2,800+ Hour Interactive World Modeling Video Dataset

udmrzn · x · 2026-08-13

Researchers introduced Sekai2, a large-scale real-world video dataset designed for interactive world modeling.

Scale: Contains 128,892 clips totaling over 2,826 hours, sourced from 10,428 videos across 113 countries/regions.
Long-form & Annotations: 43,594 clips reach the full 2 minutes under a 120-second decomposition (51.4% of total footage). Each clip includes camera trajectories and hierarchical annotations disentangling subject motion, environment dynamics, static scenes, and camera behavior, yielding 649,597 temporally grounded segments.
Key Innovation: Introduces 982 panoramic sequences captured along non-linear trajectories with loops and revisits. These provide crucial supervision for long-term spatial memory and consistent world generation.

More from Research