Track4World: HKUST and Tencent ARC's feedforward model densely tracks every pixel in 3D
rsasaki0109 · x · 2026-09-08
Researchers from HKUST and Tencent ARC Lab released Track4World, accepted to ECCV 2026, a feedforward model that tracks the 3D trajectory of every pixel in monocular video.
- Prior methods are limited to sparse first-frame points or slow optimization-based dense tracking; Track4World performs efficient holistic tracking in a world-centric coordinate system
- Built on a VGGT-style ViT global 3D scene representation, it applies a novel 3D correlation scheme to jointly estimate pixel-wise 2D and 3D dense flow between arbitrary frame pairs
- Also supports camera-centric tracking by lifting 2D correspondences into 3D space
- Outperforms existing methods across 2D/3D flow estimation and 3D tracking benchmarks, targeting real-world 4D reconstruction
Code, arXiv paper, and demo are available.
More from Research
- Mitra-v2: Synthetic-Data-Only 77M Tabular Model Matches 1.6B Rivals on TabArena — chaumian · 2026-09-08
- Sony AI's Hakken system turns 1.5M aging hypotheses into 2 confirmed gene discoveries — i_dg23 · 2026-09-08
- New worklog details building an async RL framework from scratch in JAX, from multi-actor systems to weight sync — yoshiyama_akira · 2026-09-08
- TASTE: A New Benchmark Testing If Models Can Predict AI Safety Researchers' Preferences — burny_tech · 2026-09-08
- EMNLP 2026 Paper: Training World Models for Behavior Consistency Cuts False Positives from 42.5% to 9.5% — 机器之心 · 2026-09-08
- Paper: skill retrieval gains can hide worse performance on the very tasks retrieved — rohanpaul_ai · 2026-09-08