FourTune: True 4-bit Diffusion Post-Training

amaarora · x · 2026-07-19

FourTune addresses the fact that diffusion model post-training doesn't truly become 4-bit just by quantizing weights to 4-bit, as activations and backward gradients still dominate compute and memory overhead.

It proposes three design choices to stabilize and accelerate training:

A frozen stabilizer branch to handle quantization-sensitive outliers;
Block quantization to enable transposed matrix multiplication in 4-bit backpropagation;
Fused kernels to reduce memory traffic from small branches.

Experimentally, the authors report that on FLUX customization tasks, the method achieves 2.25x lower memory and 2.27x faster stepping compared to BF16 LoRA. The accompanying charts compare training latency, memory footprint, and customization quality across different settings.

Original post →

FourTune: True 4-bit Diffusion Post-Training

More from Multimodal