Swift-Image: Exploring the Performance Frontier of Compact Unified Image Generation Models
Taihang Hu, Zhao Wang, Zuan Gao, Tao Liu, Hao Yan, Zhengze Xu, Yuhang Yu, Yongchao Du, Xingjian Wang, Jun Zheng, Qinye Zhou, Zhengrui Chen, Chao Lin, Yefeng Shen, Zhengtao Wu, Ge Wu, Xiaoli Xu, Denghui Yang, Huayu Zhang, Mingzhou Zhang, Mengting Chen
cs.CV
2026-08-21
Alibaba's 6B single-stream DiT unifies T2I and image editing in 243K GPU hours. With API Prompt Enhancer it scores 4.41 overall, beating 20B Qwen-Image-Edit; 3B barely drops.
Open image models have improved on fidelity, text rendering, and instruction edits. They have also grown into the teens or twenty billions of parameters, with generation and editing often split into separate checkpoints. The question here is narrower: under a hard compute budget, how far can a compact unified model be pushed by training engineering rather than by a much larger backbone.
One set of weights has to do text-to-image, single-image editing, and multi-image editing. Those jobs pull against each other. Generation wants coverage and diversity. Editing wants faithful use of references and strict preservation of untouched regions. Harder prompts add text rendering, layout, and domain knowledge. Small models feel that conflict immediately; wasted architecture or mixed-task interference burns the whole capacity budget.
Swift-Image is from Alibaba. The renderer is a 6B parallel single-stream DiT. Conditions come from Qwen3-VL-8B: three evenly spaced hidden layers are concatenated and projected into diffusion space. The VLM's image tokens are dropped; ablations found extra sequence length with no measurable quality gain. Pixel references take a separate path through FLUX.2 AE, which also encodes the noised target.
Inside each block, attention and a SwiGLU MLP with expansion 3 run in parallel. Versus a serial block the quality gap is tiny and inference is about 10% faster. Timestep scale, shift, and gate come from one shared modulation network. That weakens modulation in isolation and frees parameters for attention and MLP. Position encoding is 4D-RoPE over time, height, width, and language, with a forced offset of 10 on the time axis between consecutive input images so same-location tokens do not collapse into copy-paste. Quoted strings meant to appear in the image use character-level tokenization.
Training is a curriculum:
Post-training does not dump conflicting rewards into one policy. DiffusionNFT trains a text-to-image expert, a general editor, and several domain editors in parallel. Rewards cover alignment, aesthetics, quality, instruction following, and reference consistency, plus ArcFace for identity and PP-OCRv6 for rendered text. Classifier-free guidance is used only at rollout; the policy update itself is unguided. Multi-teacher on-policy distillation then folds the experts into one student on the student's own sampled states.
Two compressions for deployment. Structural pruning cuts attention heads from 32 to 24 to get a 3B model that inherits surviving weights, then reruns continual pre-training with a frozen 6B velocity teacher. Distribution matching distillation shortens sampling from 50 steps to 8.
At inference a Prompt Enhancer sits in front. Easy requests are rewritten into a generator-aligned visual spec. Knowledge-heavy or layout-sensitive requests first write a <think> trace; only the final refinedprompt reaches the DiT. The PE is trained with SFT then GRPO against a frozen renderer, scoring both textual correctness and whether the DiT can actually execute the rewrite.
The 6B renderer training costs about 243K GPU hours. Editing overall averages GEdit (halved from a 10-point scale), ImgEdit, REDEdit, CPI-General, and CPI-Practical.
| Model | Params | Editing overall |
| GPT-Image-2 | undisclosed | 4.61 |
| Seedream5 Pro | undisclosed | 4.57 |
| Swift-Image-6B + API PE | 6B | 4.41 |
| Swift-Image-3B + API PE | 3B | 4.40 |
| Swift-Image-6B, no PE | 6B | 4.16 |
| Qwen-Image-Edit-2511 | 20B | 4.07 |
| FLUX.2-klein-9B | 9B | 4.05 |
The 6B model without PE already edges the 20B Qwen editor. The in-house PE lifts it to 4.33; the API PE to 4.41, third among all evaluated systems, behind GPT-Image-2 and Seedream5 Pro.
CPI-Intelligent makes the PE's share obvious. Direct 6B prompting scores 2.26; PE jumps to 4.23 and API PE to 4.24. The 3B model moves from 2.02 to 4.10. GPT-Image-2 remains at 4.77.
Text-to-image depends more on the rewriter. On Qwen-Image-Bench the 6B model scores 51.10 raw, slightly below open-source Qwen Image 2512 at 52.06; PE raises it to 56.33 and API PE to 58.13. GPT Image 2 sits at 64.69. On Pi-ExpertVerse-T2I the 6B model goes from 2.49 to 4.63 (4.85 with API PE), close to GPT-Image-2 at 4.94.
Table 5 isolates post-training. Supervised 6B overall is 3.98, RL 4.16, 8-step Turbo 4.20. REDEdit moves 3.98 to 4.34; practical editing 3.69 to 3.92 to 4.01. Few-step distillation does not give back the RL gains; the aggregate even ticks up.
No single module is new, and the paper says so. Single-stream DiT, staged training, diffusion RL, prompt rewriting, pruning, and few-step distillation all exist. The transferable claim is a pipeline that makes a 6B unified model lead the open-source pack.
Three practices travel. Coverage has to come before preference; aggressive early quality filters shrink the long tail. Conflicting objectives should be specialized first and merged later; mixed-task policy optimization seesaws. A Prompt Enhancer compiles user language into the caption dialect the DiT saw in training; it is not merely a longer prompt. On knowledge tasks the PE's score gap dwarfs the gap between the 3B and 6B backbones.
The practical target is unified generation plus editing without a 16B or 20B backbone. After pruning, 3B with API PE scores 4.40 against 6B at 4.41. Turbo cuts 50 steps to 8 and editing overall moves 4.16 to 4.20.
This is systems work, not an architecture leap. Closed models still sit clearly higher: 4.61 versus 4.41 on editing overall, 64.69 versus 58.13 on Qwen-Image-Bench.
There is no Limitations section in the paper. Several caveats sit in the numbers.
"6B" does not include the condition encoder or the Prompt Enhancer. Conditioning is Qwen3-VL-8B, and the PE is a separate VLM that samples groups of rewrites. Serving cost is not one 6B DiT. The API PE is systematically stronger than the PE reported in the tables as the in-house variant: 4.41 versus 4.33 on editing, 58.13 versus 56.33 on text-to-image. Part of the open-source lead hangs on a rewriter that is not specified as shipping with the 6B weights.
CPI-Bench and Pi-ExpertVerse are in-house suites from the same group. PE gains are largest there and much smaller on public GEdit and ImgEdit. Whether the eval distribution, training data, and rewriter sit too close is not ruled out.
Without PE, 6B scores 2.26 on CPI-Intelligent, below FireRed-Image-Edit at 2.65 and Qwen-Image-Edit-2511 at 2.54. The backbone is not the knowledge reasoner; the rewriter is. Reading the unified model as geographically or historically fluent overclaims what the DiT is doing.
Few-step distillation raising the editing score is reported only on the five editing benches; few-step text-to-image numbers are absent from the main tables. The 3B "almost no loss" claim is also mostly after PE. Without PE the 3B and 6B overall scores are 4.15 and 4.16, inside noise. No GitHub is listed in the paper record. Whether weights, the 3B variant, and the PE actually ship together is not specified in the main text.