Single-Step Error Correction for Image Understanding and Generation
google · hf · 2026-07-17
## Enabling Image Understanding and Generation to Self-Correct in the Same Step This work proposes **SC-CMJP** (Self-Correcting Coupled Markov Jump Processes) and the corresponding sampler **CO₂Jump**. The core idea: image generation/understanding shouldn't be disjoint tracks, but a coupled process of "mutual correction". The authors argue that existing masked diffusion sampling methods—whether interleaved decoding or parallel updates—struggle to let both modalities check each other in real-time within the same step; and since models cannot re-mask, cross-modal conflicts are hard to fix once they occur. ### Method - Uses the transition rate of one modality to depend on the other's confidence - Introduces **remasking jump**: revokes made commitments when cross-modal evidence opposes the current decision - Based on this mechanism, proposes a training-free single-pass sampler, **CO₂Jump** ### Data & Evaluation The authors also constructed and will release three large-scale joint multimodal generation corpora: - **JEdit-1M** - **JMaze-200K** - **JNono-200K** The accompanying benchmark covers both in-distribution and out-of-distribution scenarios. Experiments show that CO₂Jump achieves the best performance in image understanding/editing and visual reasoning (like maze and Sudoku-style tasks), with performance improving as sampling steps increase, proving that cross-modal coupling benefits continuously accumulate along the trajectory.
Related event: Google Research Syncs Image Understanding and Generation(2 posts)→
More from Multimodal
- A cinematic SEEDANCE 2 prompt turns an empty sunrise city into a memory-driven video — LudovicCreator · 2026-07-21
- Krea 2 Identity Edit transfers a karate pose from a line sketch without ControlNet — NatalieCrypto · 2026-07-21
- SenseTime unveils U1 Pro and open-sources a 50M-sample vision dataset at WAIC 2026 — 机器之心 · 2026-07-21
- Anatomy of Dynamic AI Images: Subject, Environment, and Camera — GPU_FieldNotes · 2026-07-21
- MiniCPM-V 4.6 now runs locally on iPhone with no cloud dependency — amos_gyamfi · 2026-07-21
- Creator says they no longer shoot with a camera, but with prompts — taherdhanera · 2026-07-21