One-step ImageNet FID of 1.36, with the same GAN loss on 14B video

NextGAN: Towards Simple and Scalable GAN Training for Image and Video Generation

Xinhao Lou, Dongye Xie, Peng Sun, Chenyi Zhuang, Kaicheng Yu, Tao Lin

cs.CV, cs.AI

2026-11-04

NextGAN trains a scalar critic by regressing its gradient onto real-fake displacements, reaching 1.36 one-step ImageNet FID and 83.13 two-step VBench on Wan2.1-1.3B.

What problem this solves

A GAN emits a sample in one forward pass. A diffusion or flow model reevaluates a large network along a trajectory. Adversarial post-training is a common route from a pretrained generator to one-step or few-step sampling, but the critic often trains a quantity the generator does not consume.

A standard adversarial loss scores real samples and generated samples with a scalar. The generator step follows the gradient of that scalar with respect to its input. A gradient penalty only limits the norm. It does not set the direction or the magnitude.

NextGAN makes that gradient the regression target. In a frozen representation space, the real feature minus the current generated feature is a displacement. The critic matches its input gradient to that displacement at points drawn along the segment.

Method

The representer stays frozen. Inference keeps only the generator. Image runs use MAE-B/16, SigLIP2-B/16, and ResNet-18. Each encoder feeds a transformer head of four blocks that outputs a conditional scalar potential, and the head losses are summed. Off-the-shelf video encoders leaned toward static appearance and missed motion, so the video runs use Wan2.1 itself. With LoRA off, blocks 7, 11, and 15 supply text-conditioned features of clean video latents. Each of those layers has its own head, with joint attention over text and video tokens.

Pairs share a condition and are sampled independently. They are not an optimal-transport coupling. The displacement is the real feature minus the generated feature, and the query point is drawn uniformly on the segment. The loss is squared error between the critic gradient and that displacement. Endpoints are detached from the generator. After averaging by location and condition, the target is a conditional mean field. If the two conditional distributions match, swapping the endpoints reverses the displacement and leaves the interpolation law unchanged, so the mean field is zero. A generated sample is not assigned some random real sample as its destination.

The critic is restricted to a potential, so the field is a gradient and the generator can keep the ordinary GAN update: hold the critic fixed and push the potential up at generated endpoints. The two steps alternate. Once the generator moves, the next round refits the displacement on new pairs.

For a fixed generator, averaging over interpolation time makes the regression the same population objective as maximizing the potential gap between real and generated endpoints, minus gradient energy on the corridor. Minibatch updates can still differ. The adversarial penalty is squared gradient energy. WGAN-GP pulls the gradient norm toward 1. Both forms are trained on images. The video numbers come from field regression.

Image training uses two critic steps per generator step, AdamW, learning rates 1e-5 and 5e-5, and a generator EMA of 0.999. Displacements are standardized per channel, each token is rescaled to the square root of its channel count, and the regression target is the displacement divided by 5. The DiT field-regression run is 100K updates on eight H800 GPUs, 158.3 GPU-hours.

Video training follows rCM's backward simulation. Starting from pure noise, it alternates denoising and re-noising for a random rollout length, with non-increasing noise levels. The generator loss is backpropagated only through the last denoising step. One 1.3B or 14B checkpoint can then sample in 1, 2, or 4 steps. LoRA rank is 256. Reference clips were generated offline by Wan2.1-14B. Training does not query teacher scores or trajectories.

Results

On a ring of eight Gaussians, NextGAN and a non-saturating GAN share initialization, width, and a 30K-update budget. Captured mass is 92.8% versus 93.3%. Effective mode count Neff is 7.995 versus 2.770.

ImageNet 256, one step, no guidance, 50K images:

MethodSetupFID ↓IS ↑
NextGAN, regressionpixel JiT-B, 131M2.87207.1
NextGAN, regressionlatent DiT-XL/1, 675M1.54243.5
NextGAN, adversarialDiT-XL/1, 675M1.36241.7
StyleGAN-XL1 NFE2.30265.1
PaGoDA1 NFE1.56259.61
Drift1 NFE1.61307.5
TBSM1 NFE1.63243.0
FD-lossJiT-H/16, 1 NFE0.75313.0
DiT250×2 NFE2.27278.2

Baselines are taken from the original papers and may use different evaluators. A FID of 1.36 is lower than most one-step entries in the table and lower than 250-step DiT at 2.27. FD-loss is lower still, at 0.75. Latent IS is 241.7 and 243.5, under Drift at 307.5 and under the original DiT at 278.2.

Swapping the loss on JiT-B, FID-50K at 50K updates is 3.31 for adversarial NextGAN, 5.64 for a logistic GAN, 17.13 for WGAN without a gradient penalty, and 28.69 for the TBSM GAN corner. At 20K updates, ResNet-18 alone scores 5.55, MAE 7.08, SigLIP2 17.78, and all three together 4.10, 26% under the best single representer. The main-table JiT-B FID of 2.87 is the full setup. The ablation budget is shorter.

Text-to-video uses Wan2.1 and self-run VBench at 832×480 and 81 frames. One 1.3B checkpoint:

MethodOnline teacherNFETotal ↑
NextGANno180.80
rCMyes180.68
AnyFlowyes178.12
DMD2yes178.61
NextGANno283.13
rCMyes281.88
AnyFlowyes282.66
DMD2yes282.78
NextGANno482.72
AnyFlowyes483.54
Wan2.1 base—50×282.37

At the same budget, NextGAN has the highest total among the compared 1.3B methods at one and two steps. One step is 80.80, only 0.12 above rCM. Two steps reach 83.13, 1.25 above rCM and 0.47 above AnyFlow, and also above the base model at 82.37 with 100 evaluations. Four steps drop to 82.72, under AnyFlow at 83.54. Two-step quality is 83.98 and the semantic score is 79.73, still under the base semantic score of 80.22.

At 14B the totals are 79.77, 82.73, and 82.02 at one, two, and four steps, all below AnyFlow at 81.23, 83.89, and 84.10. The two-step semantic score is 82.41, below AnyFlow at 82.97. One-step 14B is also below one-step 1.3B.

Replacing the critic objective at four steps on the 1.3B video setup gives totals of 82.72 for NextGAN, 81.38 for a standard GAN, 60.71 for WGAN, and 73.07 for TBSM (GAN).

Why it matters

When a pretrained generator needs fewer sampling steps, this objective does not ask for an online teacher score or a stored trajectory. One 1.3B video checkpoint covers 1, 2, and 4 steps, and the two-step score beats rCM and AnyFlow of the same size, both of which query a teacher while training.

The generator update remains standard potential ascent. The new piece is that the critic learns its gradient by regressing displacements. The DiT field-regression fine-tune cost 158.3 H800 GPU-hours and started from an existing checkpoint.

Limitations

Proposition 1 matches conditional distributions in the chosen feature space, and only if the potential class is rich enough. Information the representer throws away is not recovered, so images or videos can still disagree. The particle-convergence argument assumes a velocity close to an oracle and does not cover alternating optimization of finite networks. Large generators are not trained from scratch. The large runs are fine-tunes, and the video runs use LoRA.

The video reference clips were generated offline by Wan2.1-14B. The training loop no longer queries a teacher, but the reference distribution still comes from a larger pretrained model. Collected real video is untested.

At 14B, every sampling budget sits under the strongest baseline in the table. Gains at that scale are not stable yet, and repeated runs are missing. On the 1.3B model, four steps score below two steps, so an extra step is not automatically better. Figures 5 and 6 are manually chosen frames. Temporal quality has to be read from the videos and from the VBench axes. Table 1 quotes other papers, so 1.36 and 0.75 are not one evaluator's numbers.

The interpolation measure moves with the generator, so the generator step is not in general the full gradient of the discrepancy DNext. Regression and adversarial critic updates match as population objectives only while the generator is held fixed.

Terms

Source

What people are saying

Related papers

All paper explainers