Zero Extra Weights: Achieving Pose and Depth Control in FLUX.2 via Reference Images

ashishsanu · reddit · 2026-08-01

Since existing ControlNets poorly support FLUX.2 klein models, a developer reconstructed a workflow to achieve structural control without extra weights.

Core Mechanism

Extracts structural maps using OpenPose, Depth Anything V2, or Canny.
Feeds the map into FLUX.2's Structure map input as a reference image (not an initial latent).
FLUX.2 attends to the reference image throughout the denoising process, maintaining structure without needing a denoise strength slider.

Details & Limitations

Zero Dependency: Requires no extra model weights across the FLUX.2 family. Preprocessing runs on CPU, consuming no VRAM.
Optional Enhancement: For tight adherence on FLUX.2 dev, users can load ControlNet Union to inject residuals, requiring significant VRAM (around 512px headroom on a 24GB card).
Limitations: Does not support stacking multiple maps (e.g., pose and depth simultaneously). Gross body pose holds well, but exact joint angles and hands may drift. Not pixel-locked.
Hardware: Klein 4B peaks at 17.9 GB VRAM for 1024² generation at bf16.

Original post →

Zero Extra Weights: Achieving Pose and Depth Control in FLUX.2 via Reference Images

More from Multimodal