Fourier Neural Operator for Parametric Partial Differential Equations
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar
cs.LG, math.NA
2020-10-18
FNO puts the integral kernel in Fourier space, beats prior learned solvers on Burgers, Darcy and turbulent NS, and evaluates in 0.005s versus 2.2s.
The expensive part of scientific computing is often not one PDE solve, it is thousands of solves over a family of coefficients: design, Bayesian inversion, turbulence closures. Finite elements and finite differences tie accuracy to grid size. Convolutional nets learn a finite-dimensional map on a fixed mesh; change the resolution and the architecture has to change. PINNs parameterize a single solution. A new coefficient means a new training run. That is still one-instance solving.
Neural operators try to learn a map between function spaces, with one parameter set across discretizations. Earlier graph integral operators used Nyström in physical space. They were expensive, and they did not converge on turbulent Navier-Stokes. The missing piece was a kernel that stays mesh-invariant and is cheap because of the FFT.
The outer skeleton is the neural operator: a pointwise lift P, stacked layers of a linear integral operator plus a pointwise nonlinearity, and a pointwise projection Q. FNO restricts the kernel to a convolution and learns a truncated complex tensor R in Fourier space: FFT, multiply the low modes by R, drop the high modes, IFFT, add a local linear map W, then activate. W is a bias that tracks non-periodic boundaries. Darcy and the time axis of Navier-Stokes are not periodic; a pure spectral multiply would lose that.
Truncation is kmax,j = 12 in 2D and 16 in 1D, with 32 or 64 channels, four layers, ReLU and batch norm. The convolution theorem turns a global integral into an O(n log n) FFT. Parameters live in frequency, so changing the grid is a change of basis, which is how zero-shot super-resolution works: train coarse, evaluate fine.
For time-dependent Navier-Stokes they try two versions. FNO-2D does a 2D Fourier layer in space and rolls time with an RNN. FNO-3D convolves in space-time and maps the whole trajectory at once. With enough data, 3D is easier to train.
Relative L2 error, 1000 train / 200 test unless noted.
Burgers (initial condition to solution at t=1):
| Method | s=256 | s=8192 |
| FCN | 0.0958 | 0.3238 |
| PCANN | 0.0398 | 0.0393 |
| GNO | 0.0555 | 0.0699 |
| FNO | 0.0149 | 0.0139 |
Darcy (coefficient to pressure): FNO sits at about 0.0108 to 0.0098. FCN degrades from 0.0253 to 0.1097. RBM is about 0.025. Error barely moves with grid size. Versus prior learned solvers, that is roughly 30% lower on Burgers and 60% lower on Darcy.
Navier-Stokes at fixed 64x64, vorticity:
| Setting | FNO-3D | FNO-2D | U-Net | TF-Net |
| ν=1e-3, N=1000, T=50 | 0.0086 | 0.0128 | 0.0245 | 0.0225 |
| ν=1e-4, N=10000, T=30 | 0.0820 | 0.0834 | 0.1190 | 0.1168 |
| ν=1e-4, N=1000 | 0.1918 | 0.1559 | 0.2051 | 0.2253 |
When data are scarce, every method sits above 15% error, with FNO-2D lowest. FNO-3D trained at 64x64x20 with ν=1e-4 and N=10000 evaluates zero-shot at 256x256x80. One forward pass on a 256 grid is 0.005 s versus 2.2 s for the pseudo-spectral solver that generated the data. An MCMC inverse problem with 30,000 forwards takes about 2.5 minutes with FNO and about 18 hours with the classical solver. Truncating a ν=1e-3 solution at 20 Fourier modes is already about 2% error; FNO with kmax=12 reaches ≤1%, because activations between layers put high frequencies back.
This is the operator skeleton behind later FourCastNet-style weather models. The object being learned is a family of PDE solution maps, not an image-to-image regressor on one grid, so the mesh can change and downstream inverse problems can backprop through the surrogate. For repeated queries, pay for data and training once, then evaluate cheaply. Turbulent-regime accuracy plus zero-shot space-time super-resolution is the capability graph neural operators did not have.
Data hunger is concrete: ν=1e-4 Navier-Stokes needs 10,000 pairs before the 3D model reaches 8% error; at 1,000 pairs everyone is above 15%. Harder PDEs may be too expensive even to generate a training set. The FFT path wants a uniform grid; non-uniform inputs fall back to a slower transform. The kernel is a translation-invariant convolution, which is the wrong parameterization on awkward geometry or high-contrast coefficients. Advection-dominated problems with discontinuities inherit the usual spectral issues. Experiments do not claim to beat a classical solver on one-shot accuracy; they claim a fast operator surrogate. Whether training finds a global minimizer is unanalyzed.