CoEvolve: a construct-then-edit post-training paradigm lets a 9B model match 241B on visual grounding
青稞AI · wechat · 2026-10-12
A detailed writeup of the CoEvolve paper on visual grounding, starting from the observation that fluent language descriptions don't equal accurate visual localization — models can explain relative positions yet still draw wrong bounding boxes.
Method: a construct-then-edit post-training paradigm.
- RER (Region-Evolution Reinforcement) organizes reasoning into explicit trajectories of "semantic explanation + region coordinates", with RL rewarding structure, spatial progress, and final grounding — no manual reasoning-chain annotation needed
- BDR (Bidirectional Denoising Refiner) reads the full trajectory, keeps the text as fixed semantic context, and predicts coordinate corrections in place, trained with a relative edit-utility preference loss that encourages beneficial and suppresses harmful edits
Results: a 9B backbone reaches 92.41% average [email protected] across eight RefCOCO splits, matching the 241B-parameter InternVL3.5-A28B (92.39%). On 30k controlled perturbed boxes, one editing round lifts mIoU by 27.4 points and fixes nearly 88% of failing boxes.
Implication: for structurally editable outputs (action parameters, code patches), train editable states, learn edit utility relative to inputs, and evaluate the state–editor combination.
More from Research
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23