TERRA: Terrain-Aware Reconstruction, Retargeting and Control for Musculoskeletal Locomotion
Merkourios Simos, Chengkun Li, Bianca Ziliotto, Alexander Mathis
cs.RO, cs.AI, cs.LG, q-bio.NC
2026-09-30
TERRA infers stairs, ramps and seats from scene-less motion capture, retargets them to a 354-muscle body, and trains one policy completing 87.6% of held-out terrain motions.
Muscle-actuated human models trained with RL can imitate hundreds of captured motions, but nearly the entire behavioral repertoire lives on flat ground. The EPFL team behind TERRA traces this to two bottlenecks. Large mocap archives like AMASS ship kinematics with no scene geometry: the subject descends a staircase that does not exist in the file. And retargeting motion onto an anatomically different muscle-driven body has to preserve contacts and respect joint and tendon limits; the penetration, foot skating and floating that flat ground forgives get amplified on stairs and ramps until the reference is simply infeasible. The closest prior system, SceneBot, builds contact-rich scenes from motion, but its code is unavailable and it is judged only by downstream tracking success, never against ground-truth geometry. TERRA chains the whole problem: reconstruct terrain from trajectories, retarget under terrain-aware constraints, then train a single muscle-actuated policy on the resulting motion-terrain pairs.
Reconstruction works from kinematics alone, no vision. Two kinds of evidence: intervals where an ankle or toe landmark stays locally stationary and near the ground are positive evidence of a support surface, while the swept trajectory of the rest of the body is negative free-space evidence that bounds what geometry can exist. Four priors (independent boxes, ramps, staircases, seats) are assembled from static MuJoCo collision boxes. Flat versus inclined comes down to two cues: ankle-toe pitch relative to a neutral calibration during stance, and the swing-foot clearance profile between consecutive stances, which is near-linear on ramps and rises early on stairs. At least three raised levels fitting every stationary foot point selects the staircase prior; otherwise terrain falls back to independent boxes. Remove those two physical cues and family classification drops from 99.9% to 73.5%.
Retargeting extends OmniRetarget's interaction-mesh objective: 17 body landmarks and sampled scene points are jointly Delaunay-tetrahedralized, with a uniform Laplacian preserving human-terrain relative geometry. TERRA adds residuals specific to a muscle-driven body: the calcaneus anchored at its touchdown point during stance, minimum separation between the legs' collision geometries, swing-sole clearance, a signed-distance term for seated pelvis contact, and explicit penalties on tendon overstretch. Each frame is solved as a sequential quadratic program over 89 generalized coordinates.
The policy is PPO on MJX-Warp: 8,192 parallel environments, 100 Hz control, observations include an 11×11 pelvis-centered height map at 0.1 m spacing. One policy, 4 billion steps, roughly 20 hours on four A100 GPUs.
Five datasets contribute 8,068 clips; the non-flat portion yields a 9.4-hour training library, with a held-out split by recorded subject (about 15%).
| Metric | Baseline | TERRA |
| Terrain-family accuracy (5,474 labeled clips) | 73.5% (ablation without physical cues) | 99.9% |
| Foot-height MAE, PRISM ground-truth meshes | 34.13 mm Voronoi, 102.41 mm plane fit | 12.95 mm |
| Ramp grade MAE at 7.5°/10° | no peer baseline reported | 0.544°/0.529° |
| Stair riser MAE at 100/170/240 mm | no peer baseline reported | 7.36/4.34/3.61 mm |
Retargeting on 6,498 motions: tendon violations 0.026% (GMR 0.556%), self-collision 0.52% (GMR 21.82%), environment penetration 0.16% versus 33.45% without the residual terms, contact-timing F1 93.80% versus 80.29% for OmniRetarget.
Downstream, held-out tracking completion reaches 87.59% from TERRA references, ahead of OmniRetarget at 80.29%, GMR at 56.43% and MuscleMimic's native fitter at 47.34%. MPJPE is 74.47 mm, slightly behind the 71.65 mm of TERRA+Voronoi, which completes fewer motions (82.16%). By terrain family: ramps 99.5%, flat 91.7%, seats 89.3%, stairs 81.9% (TERRA+Voronoi takes stairs at 85.0%), and independent boxes only 46.5%.
Policy EMG and vertical ground-reaction-force profiles are compared with held-out human recordings only as min-max-normalized waveforms, with visible similarities and mismatches; no quantitative score is reported.
Mocap archives are scene-less, and this pipeline turns their non-flat content into trainable data, 993 AMASS clips in this paper alone. The two modules also stand on their own: reconstruction runs on kinematics alone, and the contact anchors and tendon residuals can be lifted into another retargeting stack. Twenty hours on four A100s for a single terrain-general policy is an accessible budget. The honest framing: interaction meshes, SQP and PPO imitation are all established components, and the contribution is the end-to-end chain plus evaluation against real geometry. Ramps and stairs hold up well; 46.5% completion on discrete boxes says isolated footholds are still an open problem.
The authors name dynamic primitive geometry, online retargeting and moving support surfaces as open problems. Beyond that: terrain covers four static families, nothing irregular or curved; stool classification falls to 70.4% when 30% of support intervals are removed and stairs to 89.5% under 10 mm landmark noise, so accuracy depends on how densely contacts are observed. The EMG/vGRF comparison stays qualitative, so the claim of a tool for studying physiology has no quantitative backing yet. SceneBot is engaged only indirectly, through a Voronoi baseline adapted from it, because its code is closed. And the evaluation's termination bound of 0.25 m (training used 0.15 m) is a lenient success criterion to keep in mind when comparing across papers.