FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching
Thanh-Long V. Le, Steven Walton, Seunghyun Yoon, Branislav Kveton, Trung Bui, Eunho Yang, Viet Lai
cs.CV
2026-09-29
FlowTool generates retouching parameters via conditional rectified flow, beating specialized MLLM agents on reference metrics across four benches with 0.19s latency, a 50× cut.
Professional retouching maps intents such as a more cinematic look, or brighten the subject and leave the background, onto continuous sliders: temperature, shadows, contrast. The usual research recipe treats this as an MLLM agent: write a reasoning trace, pick tools, then emit numbers as discrete tokens.
That recipe fights the problem. Sliders are smooth and continuous; tokenized numbers are coarse. Autoregressive decoding also stacks errors: a bad early tool choice poisons the rest of the plan. The reasoning tokens are not the product, but they dominate latency and memory, which interactive editing cannot afford.
FlowTool, from Adobe Research and KAIST, models the editing plan as a conditional distribution over a structured continuous parameter vector. The generator is a conditional rectified flow: it learns a nearly straight velocity field from Gaussian noise to expert parameters, then integrates that field at inference.
Three pieces:
Parameters are normalized to [-1, 1]. The renderer applies tools in a fixed optimal order; the model only chooses which knobs to turn and by how much.
Training has two phases. Supervised flow matching uses 496,693 internal expert edits. Instructions are synthesized by Gemma4-31B from the source, the expert result, and the recorded parameters. The flow loss is masked to tools the expert actually used; the presence head is trained with binary cross-entropy. The backbone is frozen first so the randomly initialized DiT and presence head can settle, then LoRA adapters are opened for joint fine-tuning.
Expert parameters are one valid solution, not the only one. Reward post-training with DiffusionNFT samples a group of plans, renders them, and scores each with negative pixel L1 to the expert image plus a Qwen3.8-27B VLM-as-judge score for instruction following. Inference defaults to three ODE steps.
Baselines split into untrained proprietary MLLMs (GPT-5.6 Sol, Gemini 3.1 Pro, Claude Sonnet 5, Claude Opus 4.8) and specialized agents JarvisArt and RetouchIQ. Test beds: MMArt-Bench, a 300-example internal holdout called FlowTool-Eval, ArtEdit-Bench, and MIT-Adobe5K. Reference metrics are L1/L2 or PSNR/SSIM/LPIPS; reference-free scores (semantic consistency, perceptual quality, overall) come from GPT-5.
| Method | MMArt L1↓ | MMArt L2↓ | Adobe5K PSNR↑ |
| GPT-5.6 Sol | 10.77 | 22.68 | 17.25 |
| JarvisArt | 11.64 | 28.69 | 17.39 |
| FlowTool-SFT | 9.37 | 17.78 | 17.83 |
| FlowTool-RL | 9.42 | 17.98 | 17.90 |
Against the strongest specialized agent, L1 drops by up to 28.8% and L2 by up to 47.4%, both on FlowTool-Eval. Against the strongest generalist, L1/L2 drop by up to 13.0%/21.7%. Reference-free, FlowTool beats the specialized agents throughout. Versus proprietary models it is slightly better on perceptual quality and slightly worse on semantic consistency (MMArt SC is 2.2% behind the generalist best). RL mainly lifts instruction alignment; SC and overall rise on all three instruction-driven benches, while MMArt pixel error barely moves.
Qualitatively, the specialized agents over-edit. JarvisArt blows out a frog shot until the subject is gone. FlowTool stays closer to the human retouch and looks comparable to GPT-5.6 Sol.
Latency is the sharper gap. On 200 samples, single-example inference on an A100: FlowTool 0.19s and 9.89 GB peak memory; JarvisArt, the fastest local baseline, 9.66s and 16.13 GB. That is at least 50× faster. Proprietary models are timed via API: GPT-5.6 Sol 32.11s end to end, Claude Sonnet 5 still 11.24s.
Ablations on MMArt: drop the presence head and SC falls from 7.96 to 7.04. Skip the freeze-then-LoRA curriculum and train everything jointly from the start, SC falls to 5.99. Scaling the backbone from 0.8B to 4B helps; 9B is slightly worse (overall 8.44 vs 8.69). ODE quality jumps from 1 to 2 steps, balances best at 3, then plateaus. One step is close to plain regression and is not enough.
SFT took about 46 hours on one A100 80GB; reward training about 40 hours on eight.
Retouching is recast as conditional generation in slider space, closer to a vision-language-action setup than to a chat agent: the VLM understands, the flow model acts. If a product already has a scriptable filter stack, this avoids an agent that mangles numbers. Three ODE steps and 0.19s make live preview plausible.
This is a problem restatement plus careful engineering, not a new generative principle. Data, renderer, and tool catalog stay inside Adobe. Semantic consistency still trails GPT-5.6, so instruction following has not caught the reasoning models. The fit is white-box, continuous-parameter, low-latency product retouching. Open-ended pixel editing is out of scope.
The appendix is explicit: the method assumes standardized linear ranges, on the order of [-100, 100]. Cyclic quantities such as hue, and discrete or text options, are future work.
Other gaps are quieter. Training data is internal; instructions are model-written; FlowTool-Eval is a holdout from the same pool. How much of the reference-metric gap is in-distribution fit is unclear. On the public MIT-Adobe5K split, PSNR only moves from Claude Sonnet 5's 17.58 to 17.90. Code and checkpoints need internal approval. The renderer assumes a static optimal tool order; in real software, order is part of the edit. Reference-free scores use GPT-5 as judge, same family as a proprietary baseline, with no cross-judge check.