2026-08-21
HydroGym's 61+ environments let a channel-trained policy transfer zero-shot to a Rec=200k 3D wing, cutting local skin friction 38% and total drag 11% at 10,000× lower cost.
Fluid control sits at the junction of ugly physics and ugly compute. The dynamics are high-dimensional, nonlinear, and multiscale; classical closed-loop methods have spent decades on turbulence and still cannot treat it as a routine engineering problem. The industrial numbers in the paper are not decorative: active drag reduction is cited as a path to 15% less aviation fuel, coordinated wind-farm control as 4-5% more output.
Reinforcement learning scaled in protein folding and tokamak plasma control because those communities shared environments. Fluids did not. A typical controller in the literature is tuned to one geometry, one operating point, and one mesh. Move the Reynolds number and it fails. The paper names this a specificity trap. Every environment step is a CFD solve, model-free RL wants thousands to millions of steps, and large-scale flow control stays both too expensive to train and too idiosyncratic to compare.
HydroGym is built to close that gap: a common suite of environments, and a transfer experiment that asks whether a policy found on a cheap surrogate can be dropped onto a simulation that would otherwise be too expensive to explore.
A University of Washington group led by Steven Brunton, with RWTH Aachen, Technical University of Munich, KTH, and the University of Michigan, casts flow control as a discrete-time Markov decision process and hides the CFD behind Farama's Gymnasium API. Switching solvers is one import line. Six backends cover different regimes: Firedrake finite elements for 2D prototyping; m-AIA lattice-Boltzmann and finite-volume codes for large DNS; Nek5000 spectral elements for high-order incompressible flow; JAX and JAX-Fluids for end-to-end differentiable solvers whose gradients run through the full trajectory.
The release is 61-plus validated environments, from laminar cylinders to a Rec = 4×10^5 three-dimensional wing, in two and three dimensions. The largest case, a DRA2303 external flow, uses 1.6 billion cells. Actions are temporally smoothed so the solver does not see discontinuous boundary conditions. Observations and action bounds are normalized to [-1, 1], so off-the-shelf PPO, DDPG, and TD3 are usable baselines without per-environment hyperparameter hunts.
Three extras match three bottlenecks.
Gradient-enhanced PPO (GPPO) on differentiable environments folds the analytic gradient of the reward trajectory into PPO's clipped surrogate, cutting the variance of likelihood-ratio estimates. A multi-agent setup splits 3D cylinders, channels, and wings into spanwise pseudo-environments that share a policy and a replay buffer, so distributed actuation does not explode the action dimension. The headline experiment is physics-guided zero-shot transfer. The working hypothesis is that a turbulent channel and an airfoil suction side share near-wall streak and streak-vortex dynamics, even when the outer geometry, pressure gradient, and Reynolds number do not. Training happens only in a channel at Reτ = 206: multi-agent TD3, observations restricted to wall-tangential and wall-normal fluctuations at y+ = 15, actions as wall-normal blowing and suction bounded by the friction velocity and constrained to zero net mass flux, reward equal to one minus relative wall shear. The frozen policy is then deployed, with no on-wing training, onto the suction side of a 3D NACA0012 at Rec = 200,000 and zero angle of attack. Observations are rescaled by the local viscous units; the action update rate tracks the local friction velocity.
Training every validated baseline took more than 150,000 GPU hours. Across environments the agents keep rediscovering the same physical moves: energize the boundary layer, break acoustic feedback, reorganize the wake.
| Setup | Result | Note |
| 2D fluidic pinball, Re = 100 | 90% drag cut | Counter-rotating rear cylinders |
| 3D gusting airfoil, Re = 1000, α = 20°, gust ratio 2 | 20% lower load oscillation | Three leading-edge jets |
| 3D cylinder MARL, Re = 3900 | 8% drag cut | Matches prior Suárez et al. MARL |
| Differentiable Kolmogorov flow | GPPO uses at least 65% fewer training iterations than PPO | Smaller action amplitudes too |
| Fine-tune across Re, geometry, 2D to 3D | About half the episodes to converge | 2D to 3D saves 50-60% compute |
The number the paper is written around is the zero-shot wing. A channel-trained policy on a Rec = 200,000 NACA0012:
| Method | Skin-friction drag Cd,f cut | Total drag Cd cut |
| TD3 zero-shot | 15.2% | 11.0% |
| Opposition control | 7.6% | 6.8% |
| Uniform blowing | 3.8% | 0% |
Local skin friction drops 38%. The channel surrogate has two orders of magnitude fewer grid points than the wing; counting time as well, exploration cost falls by more than 10^4. Even with periodic on-wing validation, wall-clock time is still about 60 times lower than training on the wing.
The 90% figure lives on the easiest 2D pinball. The 38% figure lives on a transfer that shares near-wall physics. Neither is a universal fluid-control result.
For scientific RL, HydroGym is the closest thing fluids have to MuJoCo: one API, a ladder of environments, and published baselines. Papers can finally be compared on the same tasks. For flow-control engineers, the transfer path is the useful part. Searching a policy directly on a Rec = 200,000 3D wing is already expensive; the paper estimates that embedding a full-aircraft solve in an RL loop would take more than 500 GPUs and 50 TB of memory per training run. Learning near-wall control in a cheap channel, then pasting the policy on, is what makes hyperparameter search and multi-agent coordination feasible.
This is infrastructure plus a clean proof of concept. It is not active drag reduction on a flight vehicle. Code, Docker images, and a subset of trajectories are public.
The authors draw the boundary themselves. Every environment is DNS, with no turbulence model. Internal flows reach Reτ = 2,200 and external aerodynamics Rec = 400,000; industrial flight is 10^7 to 10^8. Zero-shot transfer is conditioned on shared near-wall turbulence physics. Whether it survives strong adverse pressure gradients, separation, or compressibility is explicitly left open. Current cases lean toward canonical geometries and near-equilibrium boundary layers. Automatic differentiation through chaotic 3D trajectories remains numerically fragile. A single large flow field is 75 GB; a 200-step test trajectory is 15 TB, so full training dumps cannot be released.
Two discounts belong on the reader side. "Drag reductions exceeding 90%" will be quoted as a platform headline; it is a Re = 100 two-dimensional pinball. The wing experiment is a symmetric section at zero incidence, with a moderate adverse pressure gradient treated as one continuous control block. That is not a cambered, lifting, spanwise-varying wing. The authors sketch a foundation model for flow control. What they have shown is that pretraining is meaningful inside a family that shares the same near-wall physics.