ReferTrack tracks language-specified targets with one camera and reaches 89.4% on EVT-Bench
tencent · hf · 2026-07-24
ReferTrack proposes a referring-then-tracking pipeline for embodied visual tracking.
- It tackles EVT with a single forward-facing camera.
- The model first selects the target from indexed bounding boxes, then predicts tracking waypoints.
- To preserve motion cues, it keeps a sliding window of previously selected boxes and injects their geometry through TVBI tokens.
- It also co-trains on a custom Refer-QA dataset to improve target identification.
On EVT-Bench, it reports state-of-the-art single-view success rates of 89.4%, 73.3%, and 74.1% on the single-target, distracted, and ambiguity splits.
The authors also say real-world deployments on legged and humanoid robots show strong sim-to-real transfer. Code is available on GitHub.
More from Embodied
- RealSense D585 Pro AI-Powered 3D Stereo Depth Camera Unboxed for Physical AI — chrismatthieu · 2026-10-26
- Physical world agent swarm demo shared on X — pduan · 2026-10-26
- Looking for a wearable mic with best battery life that feeds recordings to personal agents via MCP — altryne · 2026-10-26
- 'Lamp Swarm': a desk of lamp-shaped robots shows what physical AI work looks like — dee_hw · 2026-10-26
- Tencent's HiSTEVE 1.0: hierarchical VLA agent plays Minecraft with native keyboard and mouse, hits 85% SOTA — _akhaliq · 2026-10-26
- SJTU's VaM lets robots learn online from mistakes, 95%+ success over 4-hour autonomous runs — chris_j_paxton · 2026-10-26