Pixel-space image generation hits FID 1.45 by letting the flow-matching endpoint evolve by frequency

Energy-Guided Flow Matching

Haoyang Tong, Yu He, Fang Li, Lichen Ma, Jingling Fu, Dong Chen, Zhen Chen, Junshi Huang, Jie Cao

JD.com / Xi'an Jiaotong University / MAIS & NLPR, CASIA

cs.CV

2026-08-06

EG-FM replaces flow matching's fixed endpoint with a frequency-evolving one, supervising coarse-to-fine order. No backbone or inference change; ImageNet 256 FID 1.45, DPG 83.9.

What problem this solves

Pixel-space generative models skip VAE compression and build directly on pixels. That preserves high-frequency detail but forces the model to learn global structure and fine texture at once in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint along a straight line, and leaves the coarse-to-fine frequency evolution for the backbone to discover from data. Latent diffusion sidesteps the problem by discarding high-frequency components during VAE compression.

EG-FM takes the position that the coarse-to-fine ordering should not be left implicit. It should be supervised directly.

Method

The core change is one sentence: replace flow matching's fixed endpoint x with an endpoint yt(x) that evolves by frequency.

• yt(x) is the clean image passed through a heat kernel, a Gaussian low-pass filter. As t goes from 0 to 1, yt moves from a low-frequency-only global structure to the full high-frequency clean image. The sampling path becomes zt = t·yt(x) + (1−t)·ε, with a moving endpoint.
• How much high frequency should be released at a given t depends on the image's own spectral energy. Texture-rich images carry more high-frequency content than flat ones, so a single shared timeline is unfair. An energy-guided schedule computes a per-image clock h(x,t) so that the fraction of spectral energy already released stays comparable across images.
• Because the endpoint moves, the velocity target must be the exact derivative of the path, vt = yt(x) − ε + t·∂t yt(x), not the velocity field from fixed-endpoint flow matching.
• The release clock uses smootherstep (q(t) = 6t⁵ − 15t⁴ + 10t³). Its zero first derivatives at the endpoints give numerical stability, and it beats a linear clock on FID (1.99 vs 2.08).

The clean engineering property: FFT runs only at training time. At inference the backbone predicts velocity as usual, with no change in FLOPs or wall time. No backbone redesign, no data change.

Results

On ImageNet 256×256 class-conditional generation, EG-FM lowers FID across three architecturally distinct backbones, all with fewer epochs:

At a matched 200 epochs the PixelDiT-XL baseline reaches only FID 2.36; EG-FM pulls it to 1.55, beating a longer-trained baseline (320 epochs, FID 1.61) with less than two-thirds of the compute.

At 512×512, starting from the 256 checkpoint with only 40 high-resolution adaptation epochs, HyperDiT-H + EG-FM reaches FID 1.58.

For text-to-image at 512×512, EG-FM-T2I (1.3B parameters) scores 0.85 on GenEval and 83.9 on DPG-Bench. The same-architecture PixelDiT-T2I baseline scores only 0.78 on GenEval. DPG is the highest among all compared methods; GenEval is second, trailing the 1.1B DeCo-XXL by 0.01. The 12B FLUX.1-dev sits at 0.67 on GenEval.

Overhead: per-step wall time rises 0.41% to 4.81%, per-sample GFLOPs rise about 0.01% to 0.03%, and inference cost is unchanged.

Why it matters

Pixel-space generative models have recently pushed ImageNet FID into territory competitive with latent diffusion, but training cost remains their weakness. EG-FM offers a near-free training-side modification that works across three different backbones and cuts epoch counts. For teams building pixel-space generators, this is a change that drops onto an existing flow-matching pipeline without touching the inference path.

It also offers a transferable intuition: flow matching's straight-line interpolation treats all frequencies equally, but generation is coarse-to-fine. Making that prior explicit beats asking the network to learn it.

Limitations

The authors list several in the appendix. EG-FM has not been evaluated on temporally extended signals such as joint text-image modeling, video generation, or embodied decision-making. It has also not been tested at the scale of recent foundation backbones like Flux or Qwen-Image, so its scaling behavior is unverified.

One asymmetry stands out: under JiT's x-prediction parameterization, EG-FM only improves FID from 2.37 to 2.33, a gain of 0.04, far smaller than on the other backbones. The reason is that x-prediction predictions are noisy and spectrally inaccurate early in the solver, which undermines the premise of computing a per-image frequency clock.

Terms

• Flow matching: A generative framework that learns a velocity field transporting noise to data along a path, then samples by integrating that path.
• Pixel-space generation: Building a generative model directly on pixels rather than on a VAE-compressed latent, avoiding compression loss at the cost of higher dimensionality.
• Heat kernel: A Gaussian low-pass filter; convolving an image with it suppresses high frequencies and produces a smoother version.
• x-prediction: A parameterization where the network predicts the clean image x directly, as opposed to predicting velocity or noise.

Source

• arXiv 2608.05811v1
• PDF
• Hugging Face
• GitHub · https://github.com/ysng123/EG-FM

What people are saying

• EG-FM Lowers FID to 1.45 in Pixel-Space Image Generation Without Backbone Changes burny_tech

Related papers

• D-RAC turns PDFs into retrieval Markdown, cutting chunking output tokens 95.7%
• Mira-Scene places objects via pixel-aligned CCM, lifting 3D-IoU 39.8% over SAM3D
• Per-document cliff detection compresses multi-vector retrieval 75% at a 0.002 nDCG cost
• VideoGen-Agent learns tool use and lifts VABench from 56.5 to 75.6
• Tencent ARC's 5,000-hour AAA GameHorizon suite finds action planning is the real bottleneck
• WorldCrafter's pose-queryable 3D memory cuts revisit LPIPS from 0.487 to 0.255

All paper explainers