Learning Options for Compositional Motor Control with Adapter Banks
Sreejan Kumar, Marcelo Mattar, Lea Duncker
cs.LG, cs.RO, q-bio.NC
2026-09-15
A shared LSTM with a discrete adapter bank learns motor primitives that go low-rank; a novel two-petal trajectory hits hand L1 0.005 versus a teacher’s 0.04.
Skilled movement is built from a finite set of primitives that can be rescaled and reordered. In RL that idea is options: a low-level policy per option, plus a high-level chooser. Learned options often tangle. A new sequence then needs retraining, and old skills interfere.
Neuroscience has a structural proposal: motor cortex is a shared RNN, thalamic loops add low-rank perturbations, and basal ganglia pick which loop is on. Logiaco et al. spelled out that circuit, but not how to learn it end-to-end from demonstrations. Columbia and NYU turn the loop into a trainable adapter bank.
The domain is Lazzari and Saxena’s closed-loop biomechanical arm: two links, six muscles, straight reaches, clockwise and counterclockwise arcs, and sinusoids, in half and full variants, ten conditions in total. A teacher RNN sees a one-hot rule input every step. The student trains only on five compound trajectories and never sees the half-movement primitives, so any decomposition has to emerge.
The student, CABRA, uses a CompILE encoder on the teacher trajectory (observations and muscle commands, no rule input). It splits each trial into two segments, emits a distribution over K=10 latent codes per segment, and samples boundaries with low-temperature Gumbel-softmax. Codes are hard-masked into two pools: z0–z4 for extension, z5–z9 for retraction. That phase split is the only structural prior on code identity. Encoder and decoder share an LSTM core of width 256, but recurrent weights are detached on the decoder pass, so closed-loop hand L1 updates only the adapters, the muscle readout, and the input projection. Each adapter is a two-layer residual map H→H with no rank constraint.
The loss is hand-trajectory L1 plus KL on boundaries and codes, with a DIAYN discriminator and a muscle-excitation L1 as auxiliaries. Ablating both auxiliaries keeps the architectural transfer advantage over the teacher.
On the five compound movements, hand L1 sits around 10⁻² m. Reaches hug the expert; figure-eights wobble in the loop but keep the shape. A forced-code sweep shows one-to-one extension codes: Reach uses z2, clockwise circle z0, counterclockwise z3, figure-8 z1, inverse figure-8 z4. The wrong extension code raises error sharply. Retraction codes are partly shared: Reach and clockwise circle both pick z5, the others pick z7.
Geometrically, the core hidden-state manifold reaches effective rank about 30 out of 256. Each adapter’s perturbation subspace converges to rank about 5–6. The union of all ten adapters spans about 28, nearly the core manifold. Low rank is a learned solution, not an architectural cap.
Mean pairwise principal-subspace overlap is 0.69 for the teacher and 0.18 for the student. Mean DSA distance is 0.12 for the teacher and 0.25 for the student, organized by rotational sense: clockwise circle versus inverse figure-8 sits at 0.05, the two circles at 0.44.
With both networks frozen, a 10-dimensional soft policy is optimized onto a two-petal trajectory that splices pairings absent from training. Student L1 reaches 0.005; the teacher plateaus near 0.04, about an order of magnitude worse. On a harder butterfly shape the student is about 0.011 versus the teacher’s 0.058, roughly 5×. Under hard discrete selection the means are 0.018 versus 0.040, best 0.014 versus 0.036, still a 2.5× gap. Dropping DIAYN and excitation matching leaves two-petal L1 at 0.0056 and 0.0072, still below the teacher’s 0.04.
This is concrete evidence for a shared backbone plus discrete adapters in motor control: unconstrained residuals collapse to low rank, tasks occupy nearly disjoint subspaces, and a high-level policy can sequence existing codes into a new trajectory. The structure is in the same family as a LoRA library in LLMs, except the adapters are grown from demonstrations rather than grafted onto a pretrained net.
The experiment is one differentiable two-link arm and five compound movements. Treat it as a mechanism study, not a general robot skill library.
Task-identity supervision on the encoder’s second segment shapes how codes organize. One-to-one extension codes and partial retraction sharing may be products of that head. The authors note that a fully unsupervised, variable-code version could look different. Segment count is fixed at two, and the extension/retraction split is hard-masked. The teacher saw half-movements in training and the student did not, so the comparison is slightly asymmetric. Out-of-distribution tests optimize a high-level policy after freezing the net; that controller is not co-trained as hierarchical RL. Main geometry results are reported without multi-seed variance.