单卡 5090 从零训练图像模型:冻结编码器突破 VAE 模糊瓶颈

Creative-Listen-6847 · reddit · 2026-07-24

A developer is sharing their ongoing journey of training a DiT-based image generation model from scratch on a single RTX 5090. Constrained by the 32GB VRAM, training is locked at 512x512 resolution. Pushing the resolution to 1024x1024 causes severe structural breakdown—not because of the DiT, but due to the underlying VAE hitting its decoding ceiling, amplifying latent blurs and noise.

To fix this without wasting hundreds of hours of generator training, the author devised a clever workaround: freeze the VAE encoder to keep the latent space intact, and only train the decoder to output sharper images.

原文链接 →

「多模态」频道最新

更多「多模态」频道 AI 资讯 →