Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networks
Jan Schneider, Mridul Mahajan, Le Chen, Simon Guist, Bernhard Schölkopf, Ingmar Posner, Dieter Büchler
cs.RO, cs.LG
2026-04-11
GenAN learns PAM actuators from 1.4h of joint-position data and zero-shot transfers policies to PAMY2: 96% table tennis, 75% ball-in-a-cup, 90% reaching.
Tendon-driven arms with pneumatic artificial muscles (PAMs) sit the actuators in the base, so moving mass is low and collisions are softer. That should help at high speed near people. The catch is modeling: muscle force is nonlinear, hysteretic, and temperature-sensitive, and tendon friction changes with configuration. Analytic models have not been accurate enough for precise control.
Domain randomization, the usual sim-to-real patch, still needs an approximately correct plant. PAM arms do not have one, and turning the noise up tends to wreck the policy. Earlier muscle work either stayed at single-joint reaching or kept the robot real and simulated only the ball, which still costs hours of hardware time. Hwangbo-style actuator networks learn series-elastic actuators from torque labels. PAM arms typically have no torque sensors.
GenAN learns only the messy actuator. Rigid-body arm dynamics and object contact stay in an analytic simulator (MuJoCo XLA). The network maps an H-step history of joint positions and control signals to joint torques.
Data: 2500 open-loop random trajectories, 2 seconds each, about 1.4 hours total, split 80/20 by trajectory so neighboring timesteps cannot leak into validation. Commands are sampled every 0.5 s and joined with cubic splines. With no torque measurements, the torque loss uses inverse dynamics on finite-difference velocities and accelerations as labels. The position loss pushes the predicted torque through one simulator step and matches the next real position. The two errors differ by Δt² times the inverse mass matrix, so the direction of a torque error matters, not only its size.
Inputs use delta histories: keep the current value, replace past values with differences from now, then standardize, so tiny frame-to-frame changes are visible. Hwangbo used stride 4 to fight overfitting; here short strides win. The main setting is H=3, stride=1. Architecture: two hidden layers of 512, tanh, 150 epochs, about 25 minutes on an A100.
At rollout they keep an ensemble of five networks with different seeds and data shuffles, sample one member per sim step, and add a reward penalty on the ensemble output's standard deviation so the policy does not exploit model holes. Simulation runs at 500 Hz, the policy at 100 Hz. Actions are control increments, tanh-squashed then scaled by 0.01, which keeps motion near the smooth open-loop distribution the network saw.
On a held-out set of 800 trajectories, Position GenAN cuts error 6% at one step and 29% at 500 steps (1 s) versus Torque GenAN. The baseline is Fey et al.'s Unsupervised Actuator Net, which treats identification as RL. After a 500-run hyperparameter search it still lags by a wide margin. The test set was collected after the hardware transfer runs, so tendon wear already in the robot makes these numbers conservative.
Zero-shot on PAMY2, a 4-DoF tendon arm with antagonistic PAM pairs, 100 trials per task:
| Method | Reacher | Ball-in-a-cup | Table tennis |
| Position GenAN | 90% | 75% | 96% |
| Torque GenAN | 97% | 76% | 75% |
| No disagreement penalty | 86% | 56% | 82% |
| Single network | 93% | 74% | 83% |
| Action penalty 1250→250 | 70% | 55% | 54% |
Reacher success is mean error under 2° at episode end. Ball-in-a-cup success is catching the ball. Table tennis success is returning the ball to the opponent's side. The 54 g ball on a 107 g end effector is an unseen dynamic load; GenAN was trained with no ball attached.
Position GenAN beats Torque GenAN clearly only on table tennis (96% vs 75%). The other two tasks are close. A more accurate simulator does not automatically win every skill; table tennis is the one that punishes racket pose error. Dropping the action penalty makes the policy jittery and out of distribution, and all three tasks fall. Cutting the dataset to 1000 trajectories (about 33 minutes) keeps reacher success; below that, the ensemble degrades more slowly than a single net.
Actuator networks are no longer tied to torque-instrumented quadrupeds. For pneumatic, hydraulic, or tendon robots that only have joint encoders, the recipe is concrete: sweep the workspace open-loop, supervise with inverse dynamics or a one-step position loss, keep the rigid-body simulator. 1.4 hours of logging plus 25 minutes of training is cheaper than hours of hybrid real-robot RL.
The "first" claim is scoped to multi-joint muscle-actuated arms doing dynamic skills. Single-joint reaching already existed. Everything is on one robot, PAMY2. "Generalized" currently means the interface (no torque sensors), not that the same network transfers across machines.
Two limits the authors state outright. The exploration policy may need hand-tuning to cover the workspace. Tendon stretch, wear, and 3D-printed part deformation drift the dynamics, so the network needs periodic fine-tuning and cannot yet transfer across robot instances.
Ball-in-a-cup failures are mostly not the actuator: the string wraps around a link, or the ball bounces out. The MuJoCo tendon does not collide with the robot, and ball/string parameters were not randomized. On table tennis, Position GenAN's 100 trials break down as 96 success, 3 long, 0 short, 1 miss; Torque GenAN has 13 short returns and 12 misses.
A multi-step position loss takes about 12 hours on an A100 versus 25 minutes for one-step, with no consistent accuracy gain. There is no second muscle arm or hydraulic arm in the paper, so the "wide range of robots" claim is still a promise.