HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark
Dairu Liu, Zekun Qi, Jiayu Zeng, Ruixi Yu, Yu Guan, Yintianrun Zhang, Xuchuan Chen, Sikai Liang, Zekai Li, Chenghuai Lin, Xinqiang Yu, Wenyao Zhang, He Wang, Li Yi
ECCV 2026
cs.RO, cs.AI, cs.CV
2026-08-14
HumanTracker brings 153 hours of professional mocap in four motion families plus HumanScore, a preference-trained metric aligning with human judgment at 0.91 versus 0.80 for MPJPE.
Tracking a reference motion is a core capability of humanoid control; teleoperation and whole-body imitation are built on it. The ruler for measuring it is crude. The standard approach reports joint-angle errors such as MPJPE, averaging per-frame pose differences. Low error does not mean the motion looks right: foot skating, mistimed touch-downs and unstable support are exactly the artifacts human observers notice, and frame-wise averaging washes them out. The commonly used AMASS test set holds only 140 sequences, under-covering the long tail, and results often collapse into a single aggregate that hides where a controller fails.
This ECCV 2026 paper (Nankai, Tsinghua, Galbot and others) replaces both the data and the ruler: a 153-hour categorized mocap benchmark plus HumanScore, a metric trained on human preferences.
On the data side, 24 professional performers (dance teachers, fitness and tennis coaches, full-time mocap actors) recorded about 153 hours in a multi-camera optical studio. Motions were retargeted to a 29-DoF humanoid via GMR, manually screened for ground penetration, floating and broken contacts, yielding 25K clips organized into four families by failure regime: Daily 89 hours, Highly Dynamic 11, Interaction 48, Ground 5, each with text labels. The evaluation protocol standardizes the MuJoCo entry point, the qpos representation and 50 Hz logging, and keeps SONIC's whole-body termination rule: vertical error above 0.25 m at pelvis, ankles or wrists, or pelvis rotation error above 1 rad, fails the episode. Trackers keep their native observation and action stacks.
HumanScore is the centerpiece. Four existing trackers roll out on training-split motions, which are cut into 5-second windows and paired by uniform index sampling. Six doctoral researchers in humanoid robotics compare synchronized videos pairwise (or mark Similar / Cannot compare), producing 12K pairs over 24K trajectories. The scorer is a temporal Transformer over 539-dimensional per-frame features (70 reference-state dims plus 469 covering simulation state, control, measured contact dynamics and root motion), trained with a Bradley–Terry loss on strict pairs plus a symmetric constraint on Similar pairs. At inference, window rewards pass through a sigmoid to a 0–100 scale and are averaged by frame count.
Two design choices matter. The model consumes simulator trajectories rather than rendered video, which removes camera-view dependence and makes scoring reproducible. And the features include measured contact forces, which proves decisive on ground-level motions.
Zero-shot evaluation of four trackers (none trained on HumanTracker); Succ is completion rate in %, HumanScore tops out at 100:
| Method | Daily Succ/HS | Highly Dyn. Succ/HS | Interaction Succ/HS | Ground Succ/HS |
| GMT | 17.0 / 2.4 | 36.2 / 7.0 | 81.4 / 11.7 | 0.0 / 4.0 |
| TWIST2 | 60.1 / 10.1 | 39.9 / 16.9 | 91.3 / 28.3 | 0.0 / 4.5 |
| SONIC | 93.8 / 49.5 | 82.1 / 41.0 | 97.6 / 54.6 | 20.1 / 26.5 |
| Humanoid-GPT | 94.4 / 54.7 | 86.9 / 49.2 | 97.2 / 56.8 | 32.9 / 24.9 |
Humanoid-GPT is the strongest overall, leading all three metrics on Daily and Highly Dynamic. SONIC posts the best completion on Interaction and the best HumanScore on Ground, meaning its ground-level rollouts read as steadier and more natural. The Ground family breaks everyone: the best completion rate is 32.9%, so low-center-of-mass multi-contact motion is a shared weak spot.
On metric alignment, HumanScore agrees with human preferences at 0.908, ahead of every analytic diagnostic: joint-velocity error 0.840, keypoint error 0.841, MPJPE 0.805, foot-contact accuracy 0.788, joint acceleration 0.693. Ablations show that removing contact features drops Ground alignment from 0.879 to 0.759, that adding future-reference information slightly hurts, and that extending the available temporal context from 1 to 4 seconds lifts alignment from roughly 0.86 to 0.91. Artifacts that unfold over time, such as sliding and jitter, need a long enough window to be visible.
For tracker developers, the benchmark turns a single aggregate into four diagnosable failure regimes with hard numbers on Ground and Highly Dynamic. HumanScore can serve directly as a training reward, which the authors list as future work, and the preference-construction pipeline (uniform pairing across tracker rollouts, expert annotation, display-order randomization, grouping by source motion) transfers to other robotics evaluations.
For embodied AI more broadly, this is a complete template for putting human preference into robot evaluation without hand-crafting rules that enumerate artifacts.
Preference data covers only rollout distributions from four trackers; whether scores stay calibrated for trackers behaving outside that distribution is untested. The scorer reads simulator features, which removes render dependence but inserts a feature-engineering layer between it and the video-based human judgment, and whether 539 dimensions capture everything humans care about is unknown. Evaluation runs on a single 29-DoF morphology, with cross-embodiment left as future work by the authors. The four families are uneven in size (89 hours of Daily versus 5 hours of Ground); the authors explain this as natural frequency and report per family, but the Ground sample is comparatively small and its conclusions correspondingly less certain.