FourTune: True 4-bit Diffusion Post-Training
amaarora · x · 2026-07-19
FourTune addresses the fact that diffusion model post-training doesn't truly become 4-bit just by quantizing weights to 4-bit, as activations and backward gradients still dominate compute and memory overhead.
It proposes three design choices to stabilize and accelerate training:
- A frozen stabilizer branch to handle quantization-sensitive outliers;
- Block quantization to enable transposed matrix multiplication in 4-bit backpropagation;
- Fused kernels to reduce memory traffic from small branches.
Experimentally, the authors report that on FLUX customization tasks, the method achieves 2.25x lower memory and 2.27x faster stepping compared to BF16 LoRA. The accompanying charts compare training latency, memory footprint, and customization quality across different settings.
More from Multimodal
- Community-trained MOSS Sharpness Slider LoRA: 72 photos, 500 steps, +/-1 focus control — aoleg77 · 2026-10-27
- MOSS Image Edit vs Fibo-2.0-edit: 5-grid comparison of instruction-based image editing — Illustrious_Row_9971 · 2026-10-27
- Cinematic title sequence built with ComfyUI: LTX-2.5 + MiniMax H3 image-to-video — magratheya_64 · 2026-10-27
- Fish Audio ships open-weight Drama 3 TTS; creator shows turning old posts into narrated videos — alex_verem · 2026-10-27
- Redditor crafts photorealistic 'morning bus ride' short film entirely in ComfyUI — no3us · 2026-10-27
- New blur/anti-blur slider LoRA for MOSS lets you dial background sharpness with weights — NoMouse9610 · 2026-10-27