CMU's TrackEverything tracks all visible points in 1000+ frame videos within 40GB GPU memory
CarnegieMellonU · hf · 2026-09-28
CMU researchers introduce TrackEverything, a 3D point tracker that breaks the trade-off between tracking sparse points over long horizons and tracking all points only in short clips, by representing videos as persistent 3D scene tracks in world coordinates — decoupling model complexity from video duration.
- Voxel dedup: merges co-located tracks at sliding-window boundaries to prevent redundant accumulation.
- Decomposed tracking: an endpoint refiner predicts each point's destination and static/dynamic class, then a lightweight trajectory refiner decodes dense trajectories only for dynamic points.
- 3D WAFT: replaces memory-prohibitive 4D correlation volumes with efficient feature sampling in the scene cloud.
It is reportedly the first 3D tracker to track all visible points in videos exceeding 1000 frames within 40GB GPU memory, outperforming open-source dense 3D trackers by over 20% APD on TAPVid-3D short clips while staying competitive with sparse trackers on long sequences.
Related event: TrackEverything Enables Dense Long-Horizon Point Tracking in 3D Scenes(2 posts)→
More from Research
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- Kind ICLR Reviewer Raises Score Despite Unresolved Concerns, Citing the Paper's Value — MarkSchmidtUBC · 2026-11-16
- Hand-editing 0.2% of weights plus 100 CPU-minutes turns an English TTS model into an 18MB Sanskrit one — kastnerkyle · 2026-11-16
- Why chirality matters for AI for Science: enantiomers behave differently in biology — madeofAjala · 2026-11-16
- RSS 2027 submissions open with FAQ clarifying two-stage review and negative-result bar — GeorgiaChal · 2026-11-16
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16