Training-free 4.8× diffusion speedup by Chebyshev-forecasting cached features

Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration

Jiaqi Han, Juntong Shi, Puheng Li, Haotian Ye, Qiushan Guo, Stefano Ermon

CVPR 2026

cs.CV, cs.LG

2026-03-02

Caches only the last DiT block's features and extrapolates them with a Chebyshev fit; FLUX.1 runs 4.79× faster at higher fidelity than TaylorSeer, with no retraining.

What problem this solves

Diffusion samplers spend dozens of denoising steps, each a full forward pass, so generation is slow. Feature-caching accelerators exploit that features change slowly between adjacent steps: skip some forward passes and reconstruct the missing features from cached ones, cutting the effective NFE. The prior art TaylorSeer predicts skipped features with a Taylor expansion, but its error grows with step size, so aggressive skipping blurs the output. The tension is between saving little with small steps and looking bad with large ones.

Method

Spectrum's observation: within a layer, each feature channel is a smooth function of the denoising time t, well fit by a low-order polynomial. It drops Taylor (whose error scales with powers of the step size) for a Chebyshev-polynomial ridge-regression fit, whose error bound depends on the polynomial degree, not on the step size.

Concretely:

Only the input feature to the last DiT block is cached and reused across steps; the backbone is skipped entirely for those steps.
A degree-M=4 Chebyshev basis with ridge regularization λ=0.1 is fit to the past few real feature points per channel, then used to extrapolate skipped features.
An adaptive schedule grows cache intervals as r(r+1)/2·α: fine early, coarse late, since late-step features change least.

The same scheme works on DiT (FLUX, Wan, HunyuanVideo, SD3) and on U-Net SDXL, so it is tied to the "features are smooth in time" property, not to a particular backbone.

Results

Ten NFE buys roughly 4.5 to 4.8×, fourteen about 3.5×.

Model	Speedup	Quality (PSNR/dB)	TaylorSeer
FLUX.1	4.79×	22.21	17.41
Wan2.1-14B	4.67×	21.24	17.24
HunyuanVideo	4.56×	25.39	n/a
SD3.5-Large	4.32×	n/a	—

RMSE to the oracle features is 2 to 3× lower than TaylorSeer. Wan2.1-14B still scores 82.21 on VBench. The point is that it holds up at large step sizes where TaylorSeer visibly breaks down.

Why it matters

For practitioners: training-free and plug-and-play, it cuts inference cost on an existing DiT video pipeline with no retraining and no new weights. Video generation is among the most expensive inference workloads today, and this kind of method turns minutes into a minute. It is an engineering dividend, not a capability gain; the quality ceiling is unchanged.

Limitations

Training-free cuts both ways: it leans on the prior that features are smooth in time. Extreme step sizes or steps with violent feature change still extrapolate poorly; the sweet spots cluster around 10 to 14 NFE.
The PSNR comparisons are measured against an oracle reconstruction, not a FID against the real data distribution; the cost-quality tradeoff needs re-measuring on your own model.
The Chebyshev degree M and regularization λ are fixed, with no per-model adaptive recipe.

Terms

NFE (Number of Function Evaluations): count of forward passes, the cost unit in diffusion sampling; fewer is faster.
Chebyshev polynomial: an orthogonal polynomial family that minimizes the maximum approximation error on an interval, avoiding Taylor's edge blow-up.
Ridge regression: least squares with L2 regularization, preventing overfit to few sample points.
DiT (Diffusion Transformer): a diffusion model using a Transformer backbone instead of a U-Net; FLUX and Wan belong here.
Feature caching: a family of inference accelerators that skip forward passes and reconstruct missing features from history.

Source

What people are saying

Related papers

All paper explainers