Kernel-Gradient Drifting Models
Maria Esteban-Casadevall, Jorge Carrasco-Pollo, Max Welling, Jan-Willem van de Meent, Erik J. Bekkers, Floor Eijkelboom
cs.LG, math.DG
2026-05-11
Kernel gradients replace Euclidean shifts so characteristic kernels stay identifiable. QM9 spherical Laplace: 38.9% valid, 44.1% unique, no teacher.
Drifting models move the usual multi-step sampler into training. A generator maps Gaussian noise to a sample, takes one step along a kernel field, and regresses onto that point with stop-gradient, so the target does not pass gradients. Sampling is one forward pass. There is no diffusion teacher to distill. The field is attraction minus repulsion. The kernel sets how strongly samples interact. The Euclidean displacement sets the direction.
For a Gaussian kernel those two choices agree. The kernel gradient is proportional to the displacement, and the field equals τ² times the score gap between the two smoothed densities. A field that is identically zero forces the model distribution to match the data, because Gaussian smoothing is injective. A wrong distribution cannot be a fixed point.
Laplace kernels often win empirically. Turan and Ovsjanikov trace that to the spectrum: Fourier modes decay polynomially, while a Gaussian decays exponentially. For a Laplace kernel the weighted displacement is not proportional to the kernel gradient, so distant samples get amplified by their length. Figure 2 shows the arrow leaving the direction of highest density ratio as temperature rises. Identifiability fails. On a manifold that displacement is not a tangent vector.
Kernel-gradient drifting takes the direction from ∇k. Attraction is the normalized expectation of ∇k under the data. Repulsion is the same expectation under model samples. The field is the difference. A Gaussian kernel recovers the old field up to τ². Training is still one step, then regression onto the frozen landing point.
When derivatives pass under the integral, the field is the score of the smoothed density ratio, and the loss is one-step smoothed score matching (Proposition 3.1). A zero field means the smoothed densities agree. If the kernel is characteristic, smoothing is injective, so the original distributions agree (Proposition 3.3).
The attraction and repulsion coefficients have to match. Unequal coefficients make a zero field say only that the two scores are proportional, which does not give equal smoothed densities. Collapse for kernels that are not antisymmetric lines up with this condition.
On the smoothed law, the step is the Wasserstein gradient flow of KL(q̂∥p̂) (Proposition 3.4). The particles that training moves are not smoothed. A stability analysis is left open.
Kernel gradients are tangent vectors. On a manifold the update uses the Riemannian gradient and the exponential map, and the loss uses the log map, with the step inside the injectivity radius. Categorical distributions go through the Fisher–Rao isometry onto the positive orthant of a sphere, drift there, and are read back as nucleotides or SMILES.
A Euclidean translation-invariant kernel is characteristic when its Fourier transform never vanishes. A geodesic radial kernel is positive definite at every temperature only in flat space, so on a curved manifold it does not inherit the identifiability guarantee. Matérn spectral kernels with positive coefficients do. Large ν approaches the heat kernel. Spheres use spherical harmonics. Other geometries truncate the expansion.
The synthetic study uses a Laplace kernel and 3 seeds. Checkerboard and swiss roll live on S² and H² and are projected to 2D for scoring. SW is sliced Wasserstein-2, lower better. Tile is black-square accuracy, higher better.
| Setting | Euc. displacement | Euc. gradient | Manifold displacement | Manifold gradient |
| Checkerboard S² SW | 0.031±0.009 | 0.012±0.000 | 0.032±0.008 | 0.011±0.002 |
| Checkerboard S² Tile | 0.76±0.03 | 0.86±0.01 | 0.80±0.04 | 0.87±0.01 |
| Checkerboard H² SW | 0.026±0.003 | 0.025±0.001 | 0.063±0.022 | 0.037±0.013 |
| Checkerboard H² Tile | 0.79±0.01 | 0.85±0.02 | 0.80±0.02 | 0.90±0.01 |
| Swiss roll S² SW | 0.018±0.002 | 0.014±0.002 | 0.030±0.003 | 0.014±0.002 |
| Swiss roll H² SW | 0.041±0.001 | 0.026±0.000 | 0.042±0.007 | 0.044±0.015 |
Inside a fixed geometry the gradient wins almost every cell. The exception is the hyperboloid swiss roll, where the manifold gradient at 0.044±0.015 does not beat manifold displacement at 0.042±0.007. In Figure 4 the Gaussian kernel ties the two directions, and they split at ν=0.5. The text does not list C2ST values.
Earth events are scored with MMD, lower better. The first two rows are from Deng et al.
| Method | Volcano | Earthquake | Fire | Flood |
| Euclidean Laplace displacement | 0.146 | 0.044 | 0.036 | 0.064 |
| Euclidean Gaussian gradient | 0.143 | 0.047 | 0.048 | 0.072 |
| Euclidean Laplace gradient | 0.128 | 0.038 | 0.030 | 0.067 |
| Spherical Laplace displacement | 0.113 | 0.043 | 0.049 | 0.064 |
| Spherical Gaussian gradient | 0.112 | 0.158 | 0.039 | 0.058 |
| Spherical Laplace gradient | 0.113 | 0.056 | 0.047 | 0.053 |
| Spectral ν=2.5 | 0.126 | 0.037 | 0.029 | 0.070 |
The four column winners come from three configurations. Spherical Gaussian gradient, best on volcanoes at 0.112, posts 0.158 on earthquakes, worse than Euclidean displacement at 0.044. Switching Euclidean Laplace from displacement to gradient moves flood from 0.064 to 0.067. The other three columns drop.
Promoter DNA is compared at 1-NFE. The 6-mer correlation tracks local sequence statistics. Sei MSE is lower-better.
| Method | 6-mer | Sei MSE |
| G-PSD + v-pred | 0.82 | 0.035 |
| E-RMF + x-pred | 0.96±0.01 | 0.030±0.000 |
| L-RMF + x-pred | 0.88±0.00 | 0.027±0.001 |
| Euclidean displacement | 0.56±0.03 | 0.044±0.001 |
| Euclidean gradient | 0.62±0.01 | 0.032±0.001 |
| Simplex displacement | 0.70±0.10 | 0.047±0.004 |
| Simplex gradient | 0.82±0.02 | 0.045±0.004 |
| Sphere displacement | 0.78±0.08 | 0.044±0.004 |
| Sphere gradient | 0.84±0.04 | 0.043±0.002 |
Gradient beats displacement on both metrics in all three geometries. Euclidean gradient reaches Sei MSE 0.032, close to G-PSD at 0.035, with 6-mer stuck at 0.62. Sphere gradient lands at 6-mer 0.84±0.04, below E-RMF at 0.96 and S-RMF + v-pred at 0.93, and Sei MSE 0.043, worse than G-PSD at 0.035 and L-RMF at 0.027. G-ESD scores 0.13 on 6-mer. Training uses no teacher.
QM9 generates SMILES. The caption says baseline percentages come from batches of 1024.
| Method | Valid | Unique |
| UDLM+DCD | 31.5 | 31.3 |
| PairFlow+DCD | 44.3 | 44.1 |
| UDLM+ReDi | 5.8 | 5.8 |
| PairFlow+ReDi | 35.3 | 35.1 |
| Euclidean Laplace | 22.0 | 40.0 |
| Spherical Laplace gradient | 38.9 | 44.1 |
Validity rises from 22.0 to 38.9. Uniqueness rises from 40.0 to 44.1 and ties PairFlow+DCD. Validity stays under that baseline's 44.3. UDLM+ReDi sits at 5.8.
Most one-step generators are distilled from a many-step teacher. Drifting trains the one-step map directly, but the existing argument needed a Gaussian kernel and a Euclidean displacement. Kernel gradients keep that special case, make characteristic kernels identifiable, and write the update as a tangent vector. The sphere uses the exponential map. Categories enter the positive orthant through Fisher–Rao.
QM9 uniqueness ties PairFlow+DCD. DNA still trails MeanFlow. There is no image experiment. The direction fix is steadier than a change of geometry, which can get worse on earthquake data.
The conclusion states three limits. The work stays inside kernel methods, and inside Matérn kernels. It is sensitive to hyperparameters. A wider kernel class and a systematic stability study are still open, and a gap to the strongest one-step methods remains.
The smoothed-KL gradient flow describes the smoothed law. The implementation moves unsmoothed particles, and that gap is not measured. A geodesic Laplace kernel on the sphere is not positive definite at every temperature, so the flood result of 0.053, the best number in that column, sits outside the theorem. The four best MMD entries come from three different configurations.
The abstract says state-of-the-art one-step generation without distillation. On DNA, E-RMF and L-RMF are still ahead. The QM9 figures 38.9 and 44.1 have no standard deviation, and the synthetic study uses 3 seeds. Identifiability is a population statement. Whether finite samples, a truncated spectrum, and stop-gradient regression onto a moving target reach that fixed point is not shown. The step is required to stay inside the injectivity radius, and the text does not say how often that binds.
Franz et al. show the original field is the gradient of a potential only for Gaussian kernels. Cao et al. stop their divergence-functional experiments at 2D toys. The measurements here are on the sphere, on DNA, and on molecules.