ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation
Jeonghyeok Do, Seungchul Lee, Munchurl Kim
cs.CV
2026-09-01
KAIST shows latent SAR-to-EO methods inherit a poor SD2.1 VAE. A FLUX.2 codec plus a from-scratch DiT cuts SAR2Opt FID from 71.8 to 66.3 and DISTS from 0.211 to 0.185.
Synthetic aperture radar sees Earth in most weather and at night. Speckle and geometric distortion make it harder to read than electro-optical imagery. SAR-to-EO translation tries to render a readable optical appearance from a radar observation. Early GANs went unstable on texture and invented objects. Diffusion is stabler. Recent latent methods, to save compute, inherit the Stable Diffusion 2.1 VAE and almost never ask whether that codec fits radar or remote-sensing optical images.
Under latent-target training, the EO reconstruction ceiling is the decoded endpoint: encode the optical target and decode it, and that is as faithful as an exact latent prediction can look. SAR reconstruction says how much source structure remains for conditioning. Both numbers should be measured first. Common practice also concatenates SAR and noised EO at the input. Different statistics plus local pair misalignment make that an early tangle.
ReFlowSET has three parts. First, a reconstruction audit of frozen VAEs from SD2.1, SDXL, SD3.0/3.5 and FLUX.1/2 on QXS-SAROPT, SAR2Opt, SpaceNet6 and SAR-1M, training splits only. FLUX.2 wins PSNR in six of eight dataset-modality cells and lifts the mean EO ceiling 7.49 dB over SD2.1. Only that autoencoder is kept; the multi-billion-parameter generator is not.
Second, a 509.3M 24-block conditional DiT is trained from scratch in that latent space with conditional flow matching: a linear path from Gaussian noise to the EO latent, velocity as the target. The first 8 blocks keep SAR and noised EO in separate streams, then channel-concatenate into 16 joint blocks, delaying fusion. Third, a frozen DINOv3 ViT-L/16 teacher aligns noisy-EO features at the dual-stream boundary with clean EO representations via cosine distance, λ=0.5, discarded at inference. Sampling starts from noise, not from the SAR latent, so the velocity target is not an analytic function of the inputs.
QXS-SAROPT 16,001/3,999 at 256 px; SAR2Opt 1,450/627 with 512 crops. Two RTX 4090s, CFG 1.5, Euler 50 NFE. Every baseline is retrained from official code on the same splits.
Perceptual and distributional metrics are the headline; pixel-aligned scores are not.
| Method | QXS FID | QXS DISTS | SAR2Opt FID | SAR2Opt DISTS |
| SD2.1 fine-tune | 19.1 | 0.257 | 71.8 | 0.211 |
| C-DiffSET | 19.9 | 0.233 | 78.1 | 0.214 |
| ReFlowSET (SD2.1 codec) | 25.5 | 0.267 | 84.5 | 0.217 |
| ReFlowSET (FLUX.2) | 19.1 | 0.231 | 66.3 | 0.185 |
On QXS, FID ties SD2.1 fine-tuning and DISTS is best. On SAR2Opt, FID drops 7.7% from the second-best 71.8 and DISTS 12.3% from 0.211, with the best LPIPS as well. PSNR and SSIM are not first; the paper treats pair misalignment and physical ambiguity as a reason not to crown pixel metrics. Swapping only the codec inside the same framework improves all five metrics on both sets, FID 25.5 to 19.1 and 84.5 to 66.3. The generator has no pretrained SD UNet weights.
On 12-block ablations, dual-stream channel fusion moves FID 72.865 to 70.436 versus shared-stream input concat, 108M to 145M parameters. Token fusion adds almost no FID and costs memory and latency. Representation alignment moves 71.851 to 70.436 with no inference-parameter or latency change.
Latent cross-modal translation has a step people skip: measure the codec’s round-trip ceiling, then pick a latent space. A FLUX.2 encoder plus a small from-scratch DiT can beat methods that start from a pretrained SD UNet. Delayed dual-stream fusion fits a physically non-aligned pair such as radar and optical. VFM alignment is a free inference improvement. Code and weights are public.
This is a task-level incremental step, not a new generative paradigm. Anyone already doing remote-sensing translation in an SD latent should rerun that reconstruction table first.
The SAR-to-EO map is many-to-many; the model can paint optical structure the radar does not support, and the authors list suppressing those fabrications as future work. Only two paired benches; cross-sensor generalization is untested. PSNR/SSIM lag, so a downstream user who needs pixel registration rather than look will reverse the ranking. Ablations are 12-block; the main model is 24-block, and the 1:2 topology is an analogy. NFE=50 is slower than one-step E3Diff; the paper does not head-to-head latency. The FLUX.2 encoder is still natural-image pretrained; it is the better reconstruction pick, not a SAR-retrained VAE.