Editing Images into Reusable Layers via Compositional Edit-Prior Distillation
Wen Xue, Yong Liu, Xueheng Li, Si Wu, Yong Xu, Yipeng Sun, Yan Li, Simiu Gu
cs.CV
2026-10-17
LayerEditor emits an edited canvas plus ordered RGBA layers in one pass. On ScaleEdit, SSIM goes from 0.8925 to 0.8839 once stacked (~1%), vs 11.6% for Seedream edit-then-split.
Instruction-based editors return a flat image. Type, objects, and background sit in one raster, so changing a line of copy or moving a product means generating the whole frame again. Unrelated regions and edges move with it.
Decomposers such as Qwen-Image-Layered, and cascades such as Agent Banana that edit first and split second, are trained apart from the editor. The editor is judged on whether the picture looks right. The decomposer then guesses a cut. Missing pieces, duplicates, and broken opacity show up often. A usable picture can still come with layers that are hard to reuse.
LayerEditor emits two outputs in one sample: a direct edited canvas, D, and N RGBA layers ordered from back to front. N runs from 2 to 10. RGBA is color plus opacity. Alpha-over compositing on white, with opaque pixels covering whatever is below, produces the recomposed image C. Inputs are the source image, an instruction, and N. The instruction can name the content of each layer.
The backbone is FireRed-Image-Edit-1.1. Encoders and the original RGB decoder stay frozen. A rank-256 LoRA adapts the multimodal diffusion transformer. In-Context RGB-Alpha Token Modeling denoises a single token sequence that holds the canvas, every layer's RGB, and every layer's alpha, so attention can connect the final picture to each sheet. Rotary positions add a slot index s: -1 for the source, 0 for the canvas, and 2i+1 and 2i+2 for layer i's RGB and alpha. Sheets that share pixel coordinates stay distinct.
A separate alpha decoder, initialized from the RGB decoder, is trained and then frozen. A sigmoid maps it to opacity in [0, 1], and the color pathway keeps its pretrained weights. Averaging the RGB decoder's three channels instead raises RGBA L1 from 0.0350 to 0.0578 and drops the layer-segmentation score from 6.83 to 5.67.
Layer-only finetuning damages edits. Text breaks, shapes warp, and regions that should stay put get changed. Compositional Edit-Prior Distillation (C-EPD) keeps the original editor frozen as a teacher and queries it on-policy, at intermediate noise states along the student's own denoising trajectory. The teacher reads the student's current canvas latent, the source, and the instruction, then predicts an edit velocity. The student's canvas velocity is matched to that target. Each layer's clean latent is estimated, decoded, and composited back-to-front on white. That composite is matched to the teacher's decoded canvas, and the gradient returns to every layer's color and opacity. Pixels fully hidden by upper layers are still trained with flow matching, where the network predicts a velocity that carries noise toward the clean sample.
The first two epochs use flow matching alone. C-EPD is added every fifth update, with its weight rising to 0.1. Inference takes 30 steps and does not call the teacher. Aspect ratio is kept inside a 1024×1024 pixel budget.
LayerEditor-100K has 100,000 pairs: 48,329 text edits, 36,858 object edits, and 14,813 poster-level appearance edits. Each pair includes an ordered RGBA stack that rebuilds the target. Crello and PrismLayersPro keep their native layers. Flattened ScaleEdit pairs receive target layers through planning, decomposition, repair, and filtering. Crello-Testset (2,994) and ScaleEdit-Testset (1,000) use documents held out from training.
Scores center on the recomposed image C. The direct canvas D is a check on whether the edit survived the layers.
| Benchmark | Method | Direct SSIM | Comp. SSIM | Comp. FID | Alpha IoU |
| ScaleEdit | LayerEditor | 0.8925 | 0.8839 | 39.48 | 0.6802 |
| ScaleEdit | Seedream5-Pro, edit then split | 0.8933 | 0.7901 | 70.69 | 0.6327 |
| ScaleEdit | Seedream5-Pro, one call | n/a | 0.8365 | 59.62 | 0.6583 |
| Crello | LayerEditor | 0.8530 | 0.8351 | 33.86 | 0.6074 |
| Crello | Seedream5-Pro, one call | n/a | 0.7902 | 55.33 | 0.5139 |
| Crello | FireRed + Qwen-Image-Layered | 0.6491 | 0.5584 | 115.99 | 0.2208 |
On ScaleEdit, Seedream edit-then-split SSIM falls from 0.8933 to 0.7901, an 11.6% relative drop. LayerEditor falls from 0.8925 to 0.8839, a 1.0% relative drop.
On Crello, against Seedream's single call, composite SSIM rises from 0.7902 to 0.8351 (5.7% relative) and composite FID falls from 55.33 to 33.86 (38.8% relative). A Gemini-3.1-Pro rubric averages 6.00 for LayerEditor and 4.50 for that Seedream setting. Layer segmentation is 6.83 against 5.22. A Qwen-Image-Layered model supervised for editing plus layers on the same data reaches composite SSIM 0.7285.
When one edit is reorganized into anywhere from 2 to 10 layers, LayerEditor's alpha IoU is 0.6196 and its semantic-grouping score is 6.88, against 0.5703 and 5.91 for Seedream's single call. After the instruction changes, composite SSIM is 0.8765, above Seedream's 0.7914, with alpha IoU 0.5823 and RGBA L1 0.0344.
On decomposition alone, with the image unchanged and no decomposition-specific finetuning, LayerEditor reaches composite SSIM 0.8761 on 1,171 held-out Crello images, against 0.7963 for Qwen-Image-Layered. Alpha IoU is 0.6343 and the layer-semantic score is 6.39.
Removing C-EPD drops edit consistency from 6.14 to 4.67 and layer segmentation from 6.83 to 5.01. Keeping only velocity matching, or only composite matching, lifts consistency slightly over the no-distillation run and hurts visual quality, segmentation, and L1. Both terms together improve all four.
Poster and product-image revisions often touch a single layer. A flat editor cannot hand back separable type or objects, and edit-then-split tends to lose the edit it just made. LayerEditor reads the layer count and per-layer content from the instruction and returns the canvas and the stack from one sample. On top of FireRed, the new pieces are a LoRA and an alpha decoder.
The same weights can decompose an image into layers without a decomposition-specific finetune. Cascades whose editor is left unchanged sit at direct SSIM 0.6491 on Crello. Supervised finetuning on the same labels reaches composite SSIM 0.7285, still short of 0.8351.
The stack can rebuild the edit it was trained against. Whether a layer stays clean after someone drags it was not measured.
Training and evaluation stop at 10 layers. Thin structures, complex transparency, and semitransparent edges still artifact when layers move or are stacked again. Text stays rasterized, so point size and font are not editable objects. The metrics cover isolation, opacity, and in-place recomposition on the tested layouts. They do not measure reconstruction after a layer is moved. Retention of the editing prior is checked only on the constructed benchmarks, not on an independent general editing suite. Automatic labels can be wrong. Scores come from a single Gemini-3.1-Pro judge and were not compared with human ratings.
ScaleEdit's target layers were proposed by Qwen-Image-Layered and Seedream5-Pro, so part of the supervision shares a source with the baselines. Crello-Testset uses native design layers, and LayerEditor still leads there, but ScaleEdit's layer scores can favor that annotation style. The FireRed cascade comparison also mixes in the effect of training on these 100,000 pairs. Content-conditioned tests hand the model an ordered layer plan. RevealLayer receives reference boxes. Seedream returned no usable output on 5 Crello documents after API refusals, so its Crello numbers use the other 2,989. Observed latency is about 41.5 seconds against about 250 seconds per sample, on unmatched hardware, and is not a speed result.