Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer
Bo Wei, Xianhui Lin, Yi Dong, Zhongzhong Li, Zonghui Li, Zirui Wang, Jiachen Yang, Xing Liu, Hong Gu, Xiaoming Li, Wangmeng Zuo
ECCV 2026
cs.CV
2026-06-30
Two-stage DiT imitates pseudo-targets then reconstructs the real reference. On 8,573 2K images, makeup similarity is 9.22 vs 8.43 for GPT Image 1.5.
Makeup transfer paints a reference look onto a source face without moving identity or geometry. There are almost no true pairs: same person, same pose, makeup as the only change. Early systems leaned on CycleGAN-style weak priors and lost stickers and face paint. Later systems regress onto pseudo-targets from large editors. Global layout improves, and the student inherits the teacher's sins: identity drift, mushy detail, background edits. The authors call that the pseudo-target ceiling. Imitate a synthetic image and you cannot beat it.
Harbin Institute of Technology, vivo BlueImage Lab, and Nanjing University keep the pseudo-target as a placement prior and move fine texture supervision onto the real reference.
The backbone is FLUX.1-Kontext-dev with LoRA rank 32, trained by flow matching. Conditions are concatenated tokens: noisy target, source, reference. They release MakeupFaces2K (MF2K): 8,573 in-the-wild 2048×2048 portraits, 3,139 bare, 2,063 light, 1,798 heavy, 1,573 artistic, with more male faces than older sets. Stratified sampling yields 2,000 source–reference pairs; a large editor fills in pseudo-targets to make training triplets.
Stage I runs two jobs. The transfer model regresses the pseudo-target and learns semantic alignment plus global placement. An auxiliary remover R is a one-step denoiser on (pseudo-target, source) pairs with LPIPS, ArcFace identity, and landmark terms. R is then frozen and applied to the reference to get a bare-skin face Ibareref.
Stage II moves supervision onto the real reference. The pseudo-target latent is noised at σtr; one Euler step predicts a differentiable makeup carrier ẑ, never decoded to pixels. The same DiT, shared weights, reconstructs the real reference latent from the bare-skin latent plus ẑ. Reconstruction gradients flow through ẑ into the first transfer prediction, so dropped glitter, stickers, or liner make the cycle worse. Reconstruction alone can collapse to "copy the reference, ignore the source," so a bottleneck term pulls the velocity toward the pseudo-target only at σtr and keeps coarse layout. Defaults: σtr = 0.6, λrefine = 1, λbot = 0.2. Each stage is 10k steps on four H20 GPUs.
Unlike CycleGAN, the cycle reconstructs the reference, not the source. The penalty is "did the makeup travel," not "is the domain invertible."
Each of four test sets uses 100 identity-disjoint pairs; MF2K references are drawn only from artistic makeup. Makeup similarity is a 0–10 VLM score from Gemini 3 Pro (MSimG) and Qwen2.5-VL-72B (MSimQ) on 512-resized images. Identity is ArcFace cosine. Background is L2-M on the unedited mask.
MSimG is first on all four sets. On artistic MF2K it is 9.22, against 8.43 for GPT Image 1.5, 8.34 for Nano Banana Pro, and 6.73 for StableMakeup. Identity is highest on MT at 0.87. On MF2K it is 0.74, below SHMT's 0.82, but SHMT's MSimG is 3.47. GPT Image 1.5 identity falls to 0.35. L2-M is lowest everywhere: 4.31 on MF2K versus 28.32 for GPT and 13.27 for Banana. FID is worse than StableMakeup. FID tracks average-face statistics; copying true high-frequency art makeup looks out of domain.
A 21-person study, 864 ratings on a 1–5 scale: ART 3.67 makeup / 4.03 identity / 3.83 quality. Banana Pro: 3.43 / 3.12 / 3.81. GPT Image 1.5: 3.31 / 2.15 / 3.82. Humans punish identity drift harder than ArcFace does.
Ablations match the story. Stage I only: MF2K MSimG 8.82, identity 0.57, still cloning pseudo-targets. Stage II only: identity 0.70, MSimG 8.08, because the carrier has no layout. Both stages: 9.22 / 0.74, L2-M 4.31. Swap in weaker StableMakeup pseudo-targets and ART still reaches 8.56 MSimG, while retraining StableMakeup on MF2K regresses on complex makeup (6.58). The ceiling is the supervision, not only the editor. Small σtr is conservative and under-transfers; large σtr drops identity and background; 0.6 is the compromise. They train up to 2K with a wavelet high-frequency loss and call it the first high-fidelity 2K makeup transfer.
If a product uses a general editor as a makeup engine, it ships that editor's identity drift and background rewrites. ART demotes the editor to an initializer and nails detail back to a real photograph through a differentiable cycle. That is a concrete patch for pseudo-label training, not only another makeup SOTA. MF2K also gives the field a 2K, art-makeup, male-inclusive stress set. The cost is that training still needs an editor that can mint pseudo-targets, plus a trained remover. Incremental, aimed at the actual bottleneck.
Lighting and reflectance are not constrained. Hard speculars on the reference can land on a softly lit source and look physically wrong. Makeup scores come from VLMs on 512-px inputs, so 2K sticker fidelity may be partly invisible to the judge; the authors already dropped CLIP/DINO as makeup metrics because they misfire. FID disagrees with perception, and they do not win that column. One hundred pairs per set, with a narrower artistic slice on MF2K. The remover is trained on pseudo-targets, so its error on real makeup leaks into the cycle. Cross-gender and occlusion look strong in figures without a separate quantitative split. ECCV 2026; reproduction depends on access to a comparable editor for pseudo-targets.