Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning
Hengyuan Xu, Wei Cheng, Yumeng Ji, Xuanyang Zhang, Xianfang Zeng, Gang Yu, Xingjun Ma
cs.CV
2026-08-27
Aphanta scores Direct, real-edit, and ideal-reference conditions across 20 tasks. On the positive subset a Qwen pipeline rises from 0.343 to 0.445; structural edits often hurt.
Multimodal models already crop, zoom, mark, and sometimes ask a diffusion editor for a new RGB image as a visual scratchpad. A plausible edit can still miss the count, relation, symbol, or counterfactual state a task actually needs. Mixing three questions into one accuracy delta makes the conclusion drift: does an ideal intermediate even help, can the current editor draw it, and does the downstream model use the pixels.
Aphanta, from Fudan and StepFun, scopes the question to a fixed chain: query plus image, MLLM writes an edit instruction, an instruction editor renders it, the same MLLM family answers again. The paper does not ask whether diffusion models can reason. It asks on which tasks a general instruction editor is a usable visual workspace.
Each task is scored under three conditions with one shared metric:
Δedit = SB − SA is pipeline utility. Δref = SC − SA is visual headroom for that intended image. Large Δref with small or negative Δedit is a realization gap. C is a diagnostic target, not a hard ceiling, because a real edit can occasionally suit the downstream model better than the constructed reference.
Task discovery is a four-phase agent loop (GPT-5.3-Codex with human review): propose, small-set A/B/C screen, data construction, then editor training and re-evaluation. All 20 candidates stay in the audit table, including stopped and failed ones. Retained editing assets are consolidated into one Qwen-Image-Edit LoRA (AdamW, 1×10⁻⁴). One normalized evaluation row drives Direct, Actual Edit, and Reference.
A/B/C measures the pipeline. Condition B also adds a second MLLM turn and an edit-instruction text scaffold, so a gain is not proof that returned pixels were used.
Four of 20 tasks stop after the preliminary screen (repeated patterns, visual equations, gears, sparse spot-the-difference). Sixteen go forward; thirteen keep a usable pipeline. Plane-geometry auxiliary lines and flowchart routing still do not help at the final stage.
Descriptive macro-averages by dominant operation, using tasks with full A/B/C (stopped tasks keep preliminary scores):
| Operation | Direct | Actual edit | Reference | Edit Δ |
| State realization | 0.283 | 0.580 | 0.813 | +0.297 |
| Cue injection | 0.195 | 0.365 | 0.520 | +0.170 |
| Grounding | 0.673 | 0.773 | 0.758 | +0.100 |
| Structured | 0.473 | 0.404 | 0.556 | −0.069 |
Task-level numbers are sharper. Analog clocks go 0.50→0.81 (reference 0.84). Object deletion goes 0.10→0.47 while the reference already sits at 0.92, so realization still lags. Dense counting goes 0.05→0.23. Circuit diagrams fall 0.70→0.60 with a 0.88 reference. Gears fall 0.92→0.42. Flowcharts fall 0.36→0.22. Counterfactual tasks gain about 0.21 to 0.37.
On the selected positive-task subset, Qwen3-VL plus the trained Qwen-Image-Edit moves from 0.343 to 0.445 (+10.2 points, +29.7% relative), with a 0.558 reference. Within-family pairings: Seed +3.5, Gemini-3 plus Nano Banana 2 +4.5, GPT-5 plus GPT-Image-1.5 −5.0. Holding Qwen3-VL fixed and swapping only the editor: trained Qwen +10.2, GPT-Image-1.5 +8.7, FLUX.2 Klein +3.7, Seedream-4.5 −4.7. No editor wins every operation.
Transfer to BabyVision and MIRA, which Aphanta did not select, is weak. Actual edits drop all three pipelines on BabyVision. On MIRA only Gemini shows a tiny +0.0092 edit delta, while all three references beat Direct. The protocol can mark headroom; current editors do not cash it on those external suites.
The result is a task map, not a slogan that models should "think with images." Cue injection, grounding, and counterfactual state realization are the region where an editor is worth calling. Exact symbols, topology, and geometric constructions are not. The systems implication is concrete: select by task, verify after generation, drop the image when the evidence is unreliable. Failed tasks stay in the table, which is the difference between a diagnostic and a demo reel.
This is incremental measurement, not an editor leaderboard. The +10.2 on the positive subset does not generalize to an all-task score.
Phase-2 screening on headroom inflates gains on the retained subset; the paper reports the full 20-task audit separately. Operation labels overlap, so the macro-averages are not population estimates. A/B/C cannot isolate pixel causality. Needed next are matched second-pass reasoning, call-without-return, and sham-image swaps. Closed-source editors are product snapshots at evaluation time. Negative transfer on BabyVision and MIRA means the map should not be copied onto arbitrary visual-reasoning suites. Agent proposal plus human screening already filtered the task universe once.