Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network
Yulong Chen, Ziqian Zhang, Haoyu Zhang, Ao He, Senmao Li, Kai Wang
cs.CV
2026-09-18
RefineEdit freezes GRN and routes bits via signed probability drops. On nine PIE-Bench categories it hits 30.25 background PSNR versus FlowEdit's 25.03.
Text-guided image editing has to change what the prompt asks for and leave everything else alone. Training-free diffusion editors usually draw that boundary with masks, attention maps, or injected features. A tight region leaves the edit unfinished; a loose one drags the background along. Causal autoregressive editors add a second constraint: once an early token is decoded, it cannot be revised.
Generative Refinement Network (GRN) works differently. It encodes an image as Hierarchical Binary Quantization (HBQ) codes and repeatedly updates the entire binary map. At each step every bit is predicted in parallel; a cosine schedule writes some predictions back and returns the rest to the original random code. Any coordinate can be reconsidered. This paper, from City University of Hong Kong (Dongguan), CityU, and MBZUAI, treats that property as an editing interface and presents RefineEdit, a training-free prompt-to-prompt method. The claim is short: refinement is already editable.
The source image still comes from a frozen pretrained GRN (GRN-2B in the experiments), at 1024×1024 with a 64×64 spatial code grid and 50 refinement steps. Editing starts as ordinary source refinement under prompt cs. At a switch step ts the current source state is copied into an editing branch. Both branches then continue under cs and the editing prompt ce, sharing the same random code Z and the same schedule.
At each step the two branches assign probabilities to the bit the source branch just sampled. Their difference is a signed probability drop Δ: source probability minus editing probability. A positive Δ means the editing prompt no longer supports that source bit, so that coordinate is a candidate to leave the source.
Selection is two-level:
Selected bits take the editing branch's proposal. Unselected bits copy the source state at the same refinement step. That is source-anchored routing. No external mask and no attention control.
Two stabilizers keep the masks from drifting. Adaptive spatial freezing: if the mean excess response at the switch step is strong enough (default 2τspatial), the spatial mask is locked; bitwise selection keeps moving. Finite bit locking: a bit that passed both tests in any of the last K steps (default K=4) stays eligible, but its value is not frozen. It continues to be predicted and randomly refined. K=1 restores instantaneous selection.
Defaults are ts=18, τspatial=0.015, τpower=0.12. The reported numbers use per-category settings: object replacement at ts=15; object removal, background change, and style transfer at ts=10. K and the freeze multiplier are shared. The appendix says these settings were chosen from 1,499 grid runs on the evaluation set itself, with no held-out split.
Evaluation uses nine PIE-Bench categories, 560 images in total: 80 each for replacement, addition, and removal; 40 each for content, pose, color, and material; 80 each for background and style. GRN generates the source from the source prompt; every method edits that same image. Baselines emit 512×512, so GRN sources and RefineEdit outputs are resized to 512 before scoring. Grounded-SAM masks are used only for regional metrics, not inside RefineEdit.
Main table (fidelity on unedited regions; CLIP for semantics):
| Method | Backbone | Struct Dist↓ | PSNR↑ | LPIPS↓ | SSIM↑ | CLIP whole / edited↑ |
| P2P | SD 1.4 | 0.0572 | 19.02 | 0.149 | 0.771 | 24.88 / 22.26 |
| PnP-DirectInv | SD 1.5 | 0.0229 | 23.37 | 0.081 | 0.852 | 25.57 / 22.77 |
| LEDits++ | SD 1.5 | 0.0414 | 20.59 | 0.100 | 0.820 | 26.22 / 23.29 |
| FlowEdit | FLUX.1-dev | 0.0162 | 25.03 | 0.062 | 0.902 | 24.85 / 21.95 |
| RefineEdit | GRN | 0.0224 | 30.25 | 0.032 | 0.950 | 26.47 / 23.30 |
Five of seven metrics go to RefineEdit. Background PSNR is 5.22 above FlowEdit, SSIM 0.950 vs 0.902, LPIPS 0.032 vs 0.062. Structure distance still favors FlowEdit (0.0162 vs 0.0224). Edited-region CLIP is 0.01 above LEDits++. The paper treats that as comparable semantics with better preservation, not a semantic win.
The prose quotes PSNR 30.50 and whole-image CLIP 26.51, which do not match Table 1 (30.25 / 26.47). This writeup follows the table.
On one A100 at 1024 resolution, RefineEdit takes 26.77 seconds per edit, FlowEdit 27.15, PnP about 80. In a 20-person, 20-set, 400-vote blind preference test, RefineEdit received 60.25% of votes, FlowEdit 17.75%, LEDits++ 11.50%, ChordEdit 10.50%.
On the object-replacement subset, moving the switch step from 15 to 20 raises background PSNR from 26.08 to 31.14 and drops edited-region CLIP from 20.12 to 18.57: later branching keeps the background and weakens the edit. Appendix Table 6 is labeled simulated placeholders, so those ablation numbers are unused here. Qualitatively, dropping spatial freezing edits the roof around a robot; dropping bit locking leaves fingers unfinished.
This is the first training-free prompt-to-prompt editor built on GRN. Anyone already generating with GRN gets an editing path that needs no finetuning, no external mask, and no attention surgery. Localization evidence comes from bit probabilities along two refinement trajectories.
Against the diffusion and flow editing literature, the comparison is a backbone swap, not an apples-to-apples algorithm contest. The preservation numbers look strong, and part of that may come from HBQ codes being spatially aligned and copyable bit by bit. That property does not automatically transfer to SD or FLUX. Thresholds are still tuned per task category, so this is not a fire-and-forget editor.
The paper is explicit about hard limits. RefineEdit needs the GRN source trajectory; it does not take real photographs. Freezing an incomplete initial mask can lock the target out. Continued expansion can change unrelated regions. Bit locking preserves eligibility, not the intended semantic change. The switch step and both thresholds still need tuning, and the nine category settings were searched on the evaluation benchmark.
The protocol has extra caveats. Baselines run on SD 1.4/1.5, SD-Turbo, or FLUX.1-dev; RefineEdit runs on GRN-2B. Generating at 1024 and resizing to 512 to match 512 baselines can favor the higher-resolution pipeline on PSNR and SSIM. Hyperparameters were selected on the eval data, so Table 1's margin includes overfitting. The quantitative ablation table is still placeholder numbers. Directionally the preservation result is usable; the exact gap should be discounted.