One RTX 4090 and $53: a 206M text-to-image DiT pretrained from scratch hits GenEval 0.609

Asuka-FM: A System-Architectural Co-Design Framework for Efficient Text-to-Image Pretraining on Consumer Hardware

Víctor Rodrigo González, Pablo M. Olmos, Ignacio Peis, Alejandro Lancho

cs.CV, cs.AI

2026-11-20

A 206M dual-stream DiT pretrained from scratch on one RTX 4090 in 161 hours for $53, reaching GenEval 0.609 / DPG 78.51 without benchmark-specific fine-tuning.

What problem this solves

Frontier text-to-image models (FLUX.1 at 12B, Qwen-Image at 20B) are pretrained on industrial clusters, which prices academic labs out of from-scratch training. Squeeze the job onto a single 24GB RTX 4090 and three walls appear: attention cost scales quadratically with sequence length; variable aspect-ratio bucketing forces torch.compile into endless kernel recompilation, gutting real utilization; and a frozen VAE consumes roughly 60% of training wall-time when latents cannot be cached, plus over 70% of latency in few-step inference. Asuka-FM treats architecture, memory hierarchy, and training curriculum as one co-design problem.

Method

Four interlocking pieces:

Data is 625K open-domain images (125K from each of five sources) with five rotating Qwen3-VL captions per image. On the systems side, 256² latents are cached entirely in 128GB of system RAM, 512² latents stream over the network, and only SwiGLU blocks are activation-checkpointed. Sustained average MFU is 75.6%.

Results

ModelScale and computeGenEvalDPG
Asuka-5120.21B, 1× RTX 4090, 81h0.60978.51
SDXL Base2.6B, hundreds of A100s0.40672.40
SD3 Medium2.0B, large cluster0.62084.08
DeCo-XXL1.1B, BLIP3o-tuned0.86081.40
MiniT2I-L/160.91B, 8× H100, 27 days, BLIP3o-tuned0.88384.80

The cost gap is starker: Looped-DiT and MiniT2I-B/32 each cost about $2,016, PixelGen-XXL about $2,880, i1 about $380K. Asuka-FM's full latent pretraining ran 161.2 hours for $53.20. Most academic baselines fine-tune on BLIP3o-60K, a set that mirrors GenEval's prompt syntax; Asuka-FM does not, so direct GenEval comparisons flatter the baselines. On DPG its 78.51 sits in the same band as DeCo's 81.40 and clears SDXL comfortably.

One counterintuitive profiling result: on identical workloads the RTX 4090 is 1.97× faster than a datacenter A40 (308 vs 608 ms per step), driven by 72MB versus 6MB of L2 cache and higher DRAM bandwidth. Fused Triton kernels simply fit on the consumer card.

Inference reveals an amortization asymmetry: the pixel model pays its convolutional head on every step while VAE decoding is a one-time cost, putting the break-even near 12 steps. At 13 steps or more, latent sampling is 9-31% faster; at 8 steps or fewer the pixel model wins, 1.44× at 4-step distilled sampling and 3.34× at 1 step. The final pixel model reaches GenEval 0.568 / DPG 75.31, recovering over 95% of the latent model's compositional fidelity.

Why it matters

This drops the entry price of a from-scratch text-to-image foundation model from thousands of dollars on multi-GPU nodes to one RTX 4090 and about two days of cloud rental, with code, weights, and data released. The method contribution is a complete recipe where every systems bottleneck has a matching fix and the fixes interlock. Zero-shot camera control needs no extra conditioning channel, and the pixel engine matters most for few-step distillation and online RL rollouts: Flow-GRPO needs 12 VAE decodes per prompt in latent mode, and the pixel model needs none.

Framed honestly: 206M parameters on 625K images is an efficiency frontier result, not a capability leap, and it does not compete with billion-parameter models.

Limitations

The authors list three: 625K samples leave the model behind on crowd scenes and long-tail concepts; multi-word text rendering is inconsistent; pixel-mode sampling at 13+ steps is slower than latent. A few more caveats from the read: the 0.609 GenEval trails BLIP3o-tuned baselines by a wide margin, and the Pareto claim holds only at the low-compute end; training is exclusively square-cropped, so native non-square generation is untested; the pixel-adaptation stage generated its 100K synthetic support set on an A40, so a second GPU (all of $6) enters the single-consumer-GPU story; robust camera control is demonstrated mostly on the Danbooru-trained model, and the open-domain model shows geometric warping on some prompts, which the authors attribute to the square-heavy training distribution.

Terms

Source

What people are saying

Related papers

All paper explainers