Ctrl-F: Force as an Interface for Humanoid Loco-Manipulation
Mengjie Zhao, Yuxin Chen, Yiyang Shao, Chuye Hong, Jiakang Chen, Koushil Sreenath, Masayoshi Tomizuka
cs.RO, cs.LG
2026-11-04
Ctrl-F adds a 3D force per contact body on a motion tracker. Hand-task completion stays above 90% at 1.5× mass where force-blind policies hit 0; a G1 pushes and kicks 20 kg.
A general humanoid tracker can hit the reference pose and still drop the box. The clip says where the body goes. It does not say how hard to press, and a heavier load does not rewrite the clip.
HDMI, SceneBot, and LessMimic already fold object motion, per-link contact labels, or distance fields into tracking. Those signals scale off human motion, and they record when and where contact happens. Under one fixed clip there is still no knob for pushing harder. In the lift tests here, a force-blind tracker lets 1.5-6 kg boxes slide off the palms, with zero successful episodes.
Ctrl-F is two policies and one message between them: a 3D force on each chosen body. In these tasks that is the hands and one foot. The interface does not change when the task does.
The low-level tracker reads the reference, the force commands, and proprioception, and outputs joint targets. No object is present in training. A load mask marks when a body is geometrically free to push. Stance feet stay off, because they are holding the robot up. Hands stay on through most clips. The mask comes from the motion itself, with no interaction labels. When it is on, a command is sampled and an equal, opposite wrench is applied to that body. Holding the reference under that push means the policy has to produce the commanded force. With nothing to push against, exerting force is just disturbance rejection that still has to stand on the clip. The command starts at zero and ramps to the full range, 60 N at a wrist and 30 N at an ankle.
The high level trains on the frozen tracker. It sees both reference motions and the desired contact point, and it changes the command when the measured object drifts. Deployment has no force sensor and no mass. It does need object pose.
An analytical teacher supplies the first force targets. PD on the object, mapped through rigid-body dynamics, gives the wrench the object needs, plus gravity in the air or Coulomb friction on the ground. A grasp matrix splits that wrench across contact forces. A task enters this only as the grasp matrix, what each channel can physically carry, and the load. The teacher sees true mass, so it cannot be deployed, and it only has to put the policy in a sensible region. Reinforcement learning is free to leave it. Imitation follows DAgger, labeling states the current policy actually visits, and the imitation weight fades. The reward scores the object, not the robot's match to the clip. Mass is sampled at the start of an episode and can change during it.
The baseline is SONIC, a public G1 tracker, on the same clips. The ablation removes the force command and nothing else.
With the command at zero and no external load, forced-body error after root alignment is 2.96 cm against SONIC's 5.89 cm, and completion is 100% against 80.2%. Ctrl-F was trained on these clips, SONIC was not, and the G1 configuration differs. The table says the force channel is cheap when idle. It does not say the tracker is more accurate. Replaying one wrench and changing only the command, completion at full load is about 75%. Command off, sign flipped, and SONIC all sit at 30% or below, with forced-body error about three times larger. The flipped sign is worst: the policy pushes along the load.
On five hand tasks, completion with the force command stays above 90% across the mass sweep. At nominal mass, command-off completes 43% and SONIC 31%. At 1.5× both are at 0. Neither force-blind policy reaches 90% even at half nominal. The kick holds 90% out to 2.05× nominal versus 1.51× with the command removed. The sweep stays above the line past 3×; command-off crosses it at 2.11×. Large enough loads still fail.
Mass also changes inside an episode. The policy sees neither mass nor force, and the command moves with the correct sign. On tethered boxes the commanded gain is 4.7-9.2 N/kg, under the 9.81 a hanging mass needs; force delivered to the object is 9.4-10.6 N/kg. Pushing and dragging sit at 2-4 N/kg, the scale of sliding friction. A raised box gains 18.2 N/kg when mass is added and 12.9 when it is removed, summed over both palms, and about 85-90% of that is inward squeeze. A policy trained with mass fixed inside the episode adapts the same way once the object drifts.
| Task | No command | BC only | Ctrl-F |
| Raise | 19.45/0% | 26.03/6% | 4.59/100% |
| Tethered lift | 25.25/48% | 23.65/0% | 6.26/100% |
| Tethered hold | 26.40/75% | 21.05/84% | 6.28/100% |
| Push | 29.45/0% | 24.80/0% | 4.91/84% |
| Drag | 32.05/19% | 5.33/100% | 6.95/100% |
Cells are object position error in centimeters and completion. On the raise, behavior cloning commands about 37 N and about 8 N arrives on the box. The imitation loss scores the command, not what the object feels. Drag is a near-constant pull set by weight and friction, and cloning finishes it. Reinforcement learning without the teacher does not settle: on clips that alternate loaded and unloaded phases it tends to push the whole time or never, and on clips that stay loaded the task can succeed with forces that do not match the mechanics.
On a Unitree G1, object pose comes from AprilTags. Commanded pulls of 10, 20, and 30 N read 8.8, 16.7, and 24.5 N on a crane scale. Ctrl-F drags a locked-wheel cart at 14 kg and fails at 20 kg. SONIC moves only the empty cart. SONIC also fails to push a box empty or at 20 kg; Ctrl-F pushes both at similar speed. A 2 kg bin is raised in 9 of 10 trials. Kicks work empty and at 20 kg. Carrying 1 kg while kicking a cart loaded with 8 kg runs through. Adding 6 kg mid-drag onto 8 kg, the full stack keeps its pace. Locking the command about two seconds in, so only adaptation is removed, slows the walk.
General trackers are already the backbone for teleoperation and motion replay. They have no separate channel for how hard to interact, or for retuning when the load changes. Ctrl-F fixes that channel as one 3D force per contact body. The low level never sees an object, so the interface does not need a new corpus of paired interaction data. Deployment does not need a force sensor or a mass label.
What transfers is the interface and the inverse-force recipe. Hands, feet, and the combined family each have their own controller pair. A new clip means retraining. Force moves the heavy cases from outright failure to finishing at 1.5× nominal mass. Motion coverage stays narrow.
The planner needs object state. On the robot that is AprilTags, and a missed marker breaks the loop. The policies do not track an unseen clip. The paper states both.
The tracking gap versus SONIC is inflated by training on the test clips and by a different G1 setup, which the paper says outright. The scale is 12-18% short of the command, mixed with palm compliance and a cord that is not aligned with the force, so it is a magnitude check. In the mid-drag test SONIC starts from an empty cart; Ctrl-F starts at 8 kg. The teacher sees true mass only in training. The claim that a small set of interaction patterns is enough has no reported failure when the set is too small. Completion is counted on steps the robot survives, which flatters early termination. Apart from 9/10 on the 2 kg raise, the hardware runs are a small number of trials. 20 kg is the heaviest load they tried.