FuseReg: Layer-Fusion Regularization Cuts gFID up to 29% in Representation Autoencoders
USC-PSI-Lab · hf · 2026-09-28
USC PSI Lab proposes FuseReg, tackling the layer-selection trade-off in representation autoencoders (RAEs) for image generation.
- Problem: shallow encoder layers preserve pixel detail while deep layers yield better generation metrics; fixed heuristic fusion couples two stages wanting different information
- Method: train over random subsets of encoder layers; theoretically, subset sampling explicitly penalizes sensitivity to cross-layer disagreement
- Results (ImageNet-256, DINOv3-L): one decoder reconstructs from full, sparse, and single-layer fusions without retraining at higher PSNR than fusion-specialized decoders; decoder replacement alone cuts unguided gFID by 27% on an unchanged RAEv2 DiT-XL generator; extending the regularization to diffusion training cuts gFID by a further 29% on DiT-Base
Takeaway: training downstream models for layer-fusion robustness narrows the reconstruction-generation gap without touching the pretrained encoder.
More from Research
- Continual-ARC: lifelong ARC-AGI stream finds agent harness persistence rarely beats simple caching — unifyai · 2026-11-16
- DySENSE: MIT's GenAI framework synthesizes satellite imagery with dynamic city energy maps — MIT · 2026-11-16
- WebArena-Pro: a 300-task multimodal benchmark across 20 self-hosted apps for web agents — McGill-NLP · 2026-11-16
- Training-free page-sparse decode in llama.cpp: 1.72× faster 128k decoding on an RTX 5090 — topppggg · 2026-11-16
- Training Qwen3-8B to drive a simulated hand spawns a dedicated internal 'pressure' direction — repligate · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16