From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation
Xingjian Wang, Zhao Wang, Taihang Hu, Jun Zheng, Qing Jin, Qinye Zhou, Zhengtao Wu, Yongchao Du, Zuan Gao, Chao Lin, Yefeng Shen, Xiaoli Xu, Zhengze Xu, Hao Yan, Yuhang Yu, Mingzhou Zhang, Mengting Chen
cs.CV, cs.AI
2026-08-19
Alibaba reorganizes data construction around generative capabilities instead of tasks: three engines curate 440M T2I images, 120M editing pairs, and 27M image-entity pairs, and a five-stage curriculum from 256px to 1024px trains 3B and 6B unified generation-editing models from scratch.
Modern image generation pipelines (Qwen-Image, Seedream) invest heavily in data scale, filtering, recaptioning, and semantic balancing, but nearly all organize data by task: one corpus of image-caption pairs for T2I, another of triplets for editing, each optimized in isolation. This misses a fact: generative capabilities do not emerge simultaneously. They have a dependency order. Concepts learned through semantic alignment transfer to structured generation and editing, and simple content is the foundation for high-resolution complexity. A sample's value depends on which capability it targets and its relationship to other samples, not just its quality.
For generalist models that both generate and edit, the issue is central: if every task-specific dataset must independently cover all visual concepts, cost is prohibitive and supervision cannot be shared.
The framework treats data as capability-driven infrastructure with two coupled components.
Three specialized but interoperable engines. The T2I engine curates 440M images from a billion-scale pool, expands long-tail concepts, and retains a controlled fraction of defective images with defects explicitly described in captions, so the model learns to recognize and avoid them. The editing engine builds 120M pairs: beyond reversed construction with SAM3 masks for add/remove/replace, it mines editing relations from naturally associated images, including co-occurring images on the same page, adjacent video frames, e-commerce product variants, and multi-panel layouts split into before/after and bundle/component pairs that are more realistic than synthetic ones. The knowledge engine runs PageRank over 100M+ Wikidata entities to find 3M high-salience concepts, yielding 27M image-entity pairs.
Captions are the cross-task bridge. Editing instructions inherit the vocabulary and descriptive structure of T2I captions; parts of the target description preservable from a source image are replaced with [image N] references, leaving only actual changes in text. Concepts learned in T2I transfer to editing without duplicate coverage. Two caption experts initialized from Qwen3.5-27B (general and dense) are trained with SFT plus multi-dimensional reward RL; dense captions verify identities, counts, spatial relations, and OCR item by item before assembly.
A five-stage curriculum follows capability acquisition: 256px T2I pre-training for semantic coverage, 256/512px with structurally complex, knowledge-grounded, and text-rich content, joint 512px T2I-plus-editing, 512/1024px continual training tightening to high-fidelity sources, and 1024px SFT on a VLM-screened, human-reviewed subset. Task composition, concept distribution, quality, and resolution evolve together rather than as a fixed ratio. A capability-aware loop maps checkpoint failures to targeted retrieval, expert construction, and gap-aware resampling.
Two MM-DiT sizes trained from scratch. Quantitative evaluation covers only two CPI-Bench subsets:
| Model | CPI-General | CPI-Practical | Overall |
| 3B | 3.95 | 3.91 | 3.93 |
| 6B | 3.96 | 3.92 | 3.94 |
CPI-General spans 30 fundamental editing tasks (2,039 examples); CPI-Practical spans 51 real-world application types (558 examples), scored 1 to 5 by a VLM across dimensions. Qualitative comparisons sit alongside Flux.2-klein-9B, Qwen-Image-Edit-2511, FireRed-Image-Edit, and JoyAI-Image-Edit-Plus across hybrid editing, reasoning-based editing, degradation-aware restoration, and multi-image composition.
The numbers deserve honesty: the two sizes differ by 0.01, so scale sensitivity does not show; the paper provides no quantitative CPI-Bench comparison against those four competitors and no standard T2I benchmark results. What the paper actually delivers is the data engineering: panel-splitting to recover editing pairs, explicit defect annotation, and the caption bridge.
For anyone building training data at scale, this is a complete engineering demonstration of data organization as an axis orthogonal to scale: how engines divide labor, how captions align across tasks, how the curriculum is staged, and how failures flow back. Organizing by capability instead of task lets each engine cover only its own concept subset; long-tail concepts built once in T2I are reused by editing, a shape that also suits compute-limited teams. The five-stage curriculum with an evaluation feedback loop is a more reasonable skeleton for multi-task generation training than a fixed mixture.
Thin quantitative evidence is the main weakness: the core evaluation is one four-row table over two CPI-Bench subsets, and the near-identical 3B/6B scores suggest the benchmark is scale-insensitive or too small. No quantitative comparison with competitors, and no ablations of individual components (defect retention, panel mining, the caption bridge), so the causal claim that capability-driven beats task-driven is not experimentally supported; what readers see is a coherent engineering system, not a controlled comparison. The scale of 587M images makes replication implausible. Copyright and compliance of web-scraped and e-commerce sources are never discussed.