2026-09-13
HHMI and DeepMind put a 102-DoF female Drosophila in MuJoCo and train RL to imitate real flight and walking; test-set flight median CoM error is 0.25 mm.
The fruit fly is the animal whose nervous system is closest to being fully mapped, yet motor neuroscience still lacks a body that can actually move in physics. Grand Unified Fly closed the loop on visually guided flight with a simplified body and a hand-written controller. NeuroMechFly restored anatomical detail for walking and grooming, but its low-level gait was heuristic, and it did not fly.
Walking and flight share one skeleton and a completely different force environment. A single physical body is the prerequisite for asking how the same circuits switch modes. HHMI Janelia and Google DeepMind built that body: a female Drosophila melanogaster with 102 degrees of freedom inside the open-source MuJoCo engine.
The mesh comes from confocal stacks of one female. Soft tissue was digested, cuticle bleached and stained with Congo Red. Sixty-seven parts were segmented by hand, then simplified from 22.6 million vertices to about 20,000 faces and reassembled at 66 joints. Biological joints are one to three hinges; the 102 DoFs match the literature. Masses were weighed on 52 females in bulk: 0.15 mg head, 0.34 mg thorax, 0.38 mg abdomen, 0.983 mg total, 0.297 cm long, 0.604 cm wingspan.
Wings are torque-actuated; other joints are position-actuated. Tarsal and abdominal segments bend together through tendons. To walk on walls and ceilings, the authors added adhesion actuators to MuJoCo: a normal force presses the claw into the contact, widening the Coulomb friction cone. Peak adhesion per leg is about one body weight.
They skipped full CFD. A stateless quasi-steady model treats each wing as a slender ellipsoid in still incompressible fluid and approximates five effects: added mass, viscous drag, viscous resistance, Magnus lift and Kutta lift. Coefficients were tuned until hover balanced weight. Kutta lift and viscous drag dominate; the rest sit one to two orders of magnitude lower. Plus or minus 20% on those two coefficients still flew. Wings are rigid, so flexibility, vortices and turbulence enter only as time averages.
Control is imitation learning with DMPO from the acme library, an off-policy actor-critic with a distributional critic, distributed over Ray. Flight used about 10^8 simulation steps, walking about 10^9.
The flight controller is a wing-beat pattern generator plus an MLP. The WPG plays an open-loop hover pattern from D. melanogaster at 218 Hz; the MLP outputs 12 actions that add small corrections and move frequency within 10%. Legs are retracted and frozen. Input is 62-D proprioception and vestibular signals plus a high-level steering command.
Walking has no shared pattern to start from, so a plain MLP maps 286-D input to 59-D output with wings folded. Adhesion is not in the reward; the policy learns to grip on stance.
For vision, the pretrained low-level flight policy is frozen. A CNN plus a high-level navigator sit on top. Each eye is a MuJoCo camera at 32×32 with a 150° field of view, about 4.6° per pixel.
Flight references are D. hydei videos at 7,500 fps, saccades and looming evasions, 272 trajectories, about 53 s, mirrored and split 80/20 into 216 train and 56 test. Median center-of-mass error on the test set is 0.25 mm; median orientation error is under 5°. Real flies change wing-beat frequency by up to 40 Hz in maneuvers; the model moves about 0–10 Hz. Tiny left-right stroke asymmetries produce large accelerations, matching the animal.
Walking comes from 150 fps top-down video. Thirteen 2D keypoints were lifted to 3D with regularized inverse kinematics: about 16,000 snippets, 80 minutes, 13,000 train, 3,200 test. The text reports 0.4 cm median CoM error and 4° median orientation error. At 4 cm/s an average of 3.1 legs are in stance, 3.9 at 1 cm/s, never fewer than three, as in real flies. Turns shorten the inner stride and lengthen the outer. Forelimb modulation is left-right asymmetric, which real flies do not show.
| Task | Test set | Median error |
| Flight tracking | 56 trajectories | 0.25 mm position, <5° orientation |
| Walking tracking | 3,200 trajectories | 0.4 cm position (text), 4° orientation |
| Vision bumps | 1,000 episodes | 0.045 cm height, 2.2 cm/s speed |
| Vision trench | 1,000 episodes | 0.032 cm height, 0.16 cm/s speed |
Hilly terrain is impassable without adhesion. The policy grips harder with T1/T2 uphill and T3 downhill. On one Xeon E5-2697 v3 core, 10 ms of flight takes 421.5 ms to simulate, walking 68.65 ms, fast enough for RL. Code is at TuragaLab/flybody.
This is the first open whole-body fly that both walks and flies. Embodied-AI groups get a sandbox whose DoFs, actuators and sensors can be frozen or swapped. Fly circuit people get a socket for the connectome: an MLP stands in for the ventral nerve cord today, real neurons later.
The learning algorithm is not the contribution. DMPO, hierarchy and DeepMimic-style imitation were already on the shelf. The increment is the body, the fluid, the adhesion, and the datasets. Anyone who wants a general robotics benchmark still has to write the task suite.
The body is D. melanogaster; the flight kinematics are D. hydei. Baseline frequencies differ (218 Hz vs 192 Hz), and the paper does not compare wing angles quantitatively across species. Walking 3D poses are inferred from a single top view, with fixed height and pitch and sinusoidal swing arcs, so reconstruction error leaks into the reward.
The walking CoM figure is hard to reconcile with the text. Figure 3b’s y-axis tops out at 0.04 cm, while the prose says 0.4 cm median error. 0.4 cm exceeds the 0.297 cm body length. The number is recorded as printed.
Actuators are not muscles; the authors warn against reading commands as biological signals. Neck and wing-hinge joints are hinge stacks. Rigid wings plus quasi-steady flow omit flex and vortex dynamics. Legs freeze in flight, wings freeze in walking, so mode switching is untested. Whole-body muscle actuation is listed as future work and looks expensive.