FourCastNet 3: A geometric approach to probabilistic machine-learning weather forecasting at scale
Boris Bonev, Thorsten Kurth, Ankur Mahesh, Mauro Bisson, Jean Kossaifi, Karthik Kashinath, Anima Anandkumar, William D. Collins, Michael S. Pritchard, Alexander Keller
cs.LG, physics.ao-ph
2025-07-16
FourCastNet 3 trains a spherical-convolution ensemble with spectral CRPS; a 15-day 0.25° forecast takes 60s on one H100, matching GenCast at 8x speed with stable 60-day spectra.
ML weather models now beat traditional NWP on medium-range skill, with two persistent defects. Deterministic nets blur toward an ensemble mean. Diffusion or CRPS-trained ensembles pile up small-scale noise, warp spectral slopes, and can go unstable on long rollouts. GenCast gets the probabilistic structure right with denoising diffusion, at a cost: iterative sampling, a 15-day forecast in 8 minutes on a TPU v5, and a 12-hour time step.
The usual scores do not catch this. Pointwise CRPS can be minimized marginally at each grid box; shuffle ensemble members in space and the score is unchanged while the field is no longer atmosphere. RMSE can be improved by blurring. What is missing is an ensemble that is fast, calibrated, and spectrally honest.
FourCastNet 3 writes the next atmospheric state as a hidden Markov model. Inputs are the current 0.25° state, time, and spherical diffusion noise at several space and time scales. The output is the field six hours later. The training loss is pointwise CRPS plus a spectral CRPS that weights coefficients by multiplicity, so the model has to match distributions at every wavelength, not only grid-box margins.
The backbone is a purely convolutional spherical neural operator, no transformer. Global filters live in spherical-harmonic space, the convolution theorem on the sphere, in the spirit of pseudo-spectral codes such as IFS. Local filters are spherical DISCO convolutions whose kernels are linear combinations of Morlet wavelets on a disk, anisotropic enough for blocked flow around orography and adiabatic motion on tilted isentropes. Blocks follow ConvNeXt: convolution, GeLU, pointwise MLP, residual. LayerNorm is omitted because absolute magnitudes matter in physics. Empirically, four local blocks per global block work best. The encoder downsamples 721x1440 to a 360x720 Gaussian grid with embedding width 641 and does no channel mixing, so variables with very different spectra stay separate. Water channels go through a smooth spline activation that enforces non-negativity. The net predicts the next state directly rather than a tendency; the authors find this reduces high-frequency artifacts.
Training uses spatial domain decomposition for model parallelism, plus ensemble and batch parallelism, in a framework called Makani. Pretraining ran 208,320 steps on 1024 H100 GPUs, batch 16, ensemble 16, for 78 hours; then 512 A100s for 4-step autoregressive training (15 hours); then 256 H100s for a 2012-2016 finetune (8 hours). Data is ERA5, 1980-2016. The test year is 2020.
Scored in the WeatherBench 2 protocol on 12-hourly 2020 initial conditions with a 50-member ensemble:
| System | Skill | 15-day single-member cost |
| IFS-ENS | beaten by FCN3 | about 1 hour (96 CPUs, 9 km) |
| GenCast | nearly matched at medium range | 8 minutes (TPU v5, 12-hour step) |
| FCN3 | 15 of 16 scored channels match or slightly beat GenCast at short leads | 60 seconds (one H100, 6-hour step) |
Two-meter temperature is the exception; GenCast is better. The authors attribute GenCast's slight long-lead edge to the 12-hour step: fewer autoregressive evaluations, and training only on 00/12 UTC assimilation windows, when ERA5 is typically more accurate.
Calibration is close: spread-skill ratios approach 1. Rank histograms are slightly over-dispersive out to 24 hours, then under-dispersive, then flatten. Training used 16 members and evaluation 50, so larger inference ensembles are plausible. In the extra-tropical storm Dennis case, 850 hPa winds co-vary cleanly with 500 hPa geopotential, and angular power spectra keep the right slope after a 30-day rollout. Relative error versus ERA5 angular spectra stays roughly in [-0.2, 0.2]. A 60-day global forecast at 0.25°, 6-hourly, finishes in under 4 minutes on one GPU.
This is one of the few open ensemble models that is skillful at medium range, cheap at inference, and spectrally stable on long rollouts. Members are produced in one forward pass, unlike iterative diffusion, which is what you need if the ensemble has to grow to hundreds or thousands to estimate tails. Spectra that survive 60 days are the prerequisite for talking about subseasonal forecasts rather than integrating a blur into a climatology. Code is in Makani and torch-harmonics.
If the only target is medium-range gridpoint RMSE, this is incremental: skill sits next to GenCast. If the target is physical fidelity and huge ensembles, the spherical convolutions and the spectral loss are the pieces worth copying.
Precipitation is not yet a diagnostic output, and data-assimilation uncertainty is not wired in. Training data stop in 2016 while GenCast trains through 2019, so the comparison is not matched on data vintage. Two-meter temperature lags systematically, and temperature channels show bias already visible in the rank histograms. Hardware and resolution differ from both IFS-ENS and GenCast; the 8x and 60x speedups are rough. Pointwise plus spectral CRPS is still not a full multivariate scoring rule: spatial correlation is encouraged, not proved unique. The detailed case study is Dennis; systematic extreme-event statistics remain thin.