Asuka-FM: A System-Architectural Co-Design Framework for Efficient Text-to-Image Pretraining on Consumer Hardware
Víctor Rodrigo González, Pablo M. Olmos, Ignacio Peis, Alejandro Lancho
cs.CV, cs.AI
2026-11-20
A 206M dual-stream DiT pretrained from scratch on one RTX 4090 in 161 hours for $53, reaching GenEval 0.609 / DPG 78.51 without benchmark-specific fine-tuning.
Frontier text-to-image models (FLUX.1 at 12B, Qwen-Image at 20B) are pretrained on industrial clusters, which prices academic labs out of from-scratch training. Squeeze the job onto a single 24GB RTX 4090 and three walls appear: attention cost scales quadratically with sequence length; variable aspect-ratio bucketing forces torch.compile into endless kernel recompilation, gutting real utilization; and a frozen VAE consumes roughly 60% of training wall-time when latents cannot be cached, plus over 70% of latency in few-step inference. Asuka-FM treats architecture, memory hierarchy, and training curriculum as one co-design problem.
Four interlocking pieces:
Data is 625K open-domain images (125K from each of five sources) with five rotating Qwen3-VL captions per image. On the systems side, 256² latents are cached entirely in 128GB of system RAM, 512² latents stream over the network, and only SwiGLU blocks are activation-checkpointed. Sustained average MFU is 75.6%.
| Model | Scale and compute | GenEval | DPG |
| Asuka-512 | 0.21B, 1× RTX 4090, 81h | 0.609 | 78.51 |
| SDXL Base | 2.6B, hundreds of A100s | 0.406 | 72.40 |
| SD3 Medium | 2.0B, large cluster | 0.620 | 84.08 |
| DeCo-XXL | 1.1B, BLIP3o-tuned | 0.860 | 81.40 |
| MiniT2I-L/16 | 0.91B, 8× H100, 27 days, BLIP3o-tuned | 0.883 | 84.80 |
The cost gap is starker: Looped-DiT and MiniT2I-B/32 each cost about $2,016, PixelGen-XXL about $2,880, i1 about $380K. Asuka-FM's full latent pretraining ran 161.2 hours for $53.20. Most academic baselines fine-tune on BLIP3o-60K, a set that mirrors GenEval's prompt syntax; Asuka-FM does not, so direct GenEval comparisons flatter the baselines. On DPG its 78.51 sits in the same band as DeCo's 81.40 and clears SDXL comfortably.
One counterintuitive profiling result: on identical workloads the RTX 4090 is 1.97× faster than a datacenter A40 (308 vs 608 ms per step), driven by 72MB versus 6MB of L2 cache and higher DRAM bandwidth. Fused Triton kernels simply fit on the consumer card.
Inference reveals an amortization asymmetry: the pixel model pays its convolutional head on every step while VAE decoding is a one-time cost, putting the break-even near 12 steps. At 13 steps or more, latent sampling is 9-31% faster; at 8 steps or fewer the pixel model wins, 1.44× at 4-step distilled sampling and 3.34× at 1 step. The final pixel model reaches GenEval 0.568 / DPG 75.31, recovering over 95% of the latent model's compositional fidelity.
This drops the entry price of a from-scratch text-to-image foundation model from thousands of dollars on multi-GPU nodes to one RTX 4090 and about two days of cloud rental, with code, weights, and data released. The method contribution is a complete recipe where every systems bottleneck has a matching fix and the fixes interlock. Zero-shot camera control needs no extra conditioning channel, and the pixel engine matters most for few-step distillation and online RL rollouts: Flow-GRPO needs 12 VAE decodes per prompt in latent mode, and the pixel model needs none.
Framed honestly: 206M parameters on 625K images is an efficiency frontier result, not a capability leap, and it does not compete with billion-parameter models.
The authors list three: 625K samples leave the model behind on crowd scenes and long-tail concepts; multi-word text rendering is inconsistent; pixel-mode sampling at 13+ steps is slower than latent. A few more caveats from the read: the 0.609 GenEval trails BLIP3o-tuned baselines by a wide margin, and the Pareto claim holds only at the low-compute end; training is exclusively square-cropped, so native non-square generation is untested; the pixel-adaptation stage generated its 100K synthetic support set on an A40, so a second GPU (all of $6) enters the single-consumer-GPU story; robust camera control is demonstrated mostly on the Danbooru-trained model, and the open-domain model shows geometric warping on some prompts, which the authors attribute to the square-heavy training distribution.