Program-Verified Self-Evolution for Vision-Language Models
Ahmed Heakl, Sungik Choi, Moontae Lee, Salman Khan
cs.CV, cs.AI, cs.CL, cs.LG, cs.NE
2026-09-28
VQS computes answers via programs on structured parses (94.4% human-correct vs 76.4% vote) and lifts Qwen3-VL up to +3.18 on 10 benchmarks.
Self-evolving vision-language models write training questions from unlabeled images. Those questions have no gold answers, so prior loops paste a proxy: majority vote over sampled answers, or a model judge.
Human raters checked 500 generated questions. 24% of majority-vote labels and 18% of judge labels were wrong. The error also grows with the loop: VisPlay's pseudo-label accuracy falls from 72% to 61% over three rounds; R-Zero falls from 79% to 63%. Separate work has already shown that majority-vote self-reward drifts toward confident mistakes and then collapses.
If the reward is the model's own output, training cements the wrong answer.
VQS (Verifiable QA Generation for Self-Evolving Models) computes the answer from a structured parse. The pipeline, from LG AI Research and MBZUAI, has three stages.
GRPO is used in place of SFT because template prose can pull the model off its own distribution. The parser is trained without structure labels. For each image the system samples K=4 parses, scores each by the fraction of verified claims, and keeps the best only if it beats the mean by δ=0.10 and asserts at least m=4 claims. That rule turned 12,000 unlabeled images into 4,792 SFT targets. The vision tower stays frozen. Parser, checker, blind gate, and solver share one set of weights.
A single yes/no on a whole parse cannot tell one error from five. Short claims can.
Training uses the unlabeled images from Venkatraman et al. and throws away their questions and labels. Evaluation is the lmms-eval default on ten benchmarks.
| Scale | Base avg | Strongest baseline VISE | VQS |
| 2B | 59.22 | 60.23 (+1.01) | 62.40 (+3.18) |
| 4B | 66.70 | 67.74 (+1.04) | 69.01 (+2.32) |
| 8B | 68.81 | 69.47 (+0.66) | 71.24 (+2.43) |
The largest 2B jumps sit on tasks that read a figure and combine facts: MMMU 38.92 to 46.78 (+7.86), ScienceQA 79.42 to 86.81 (+7.39). The single biggest lift is 4B MMMU, 45.17 to 54.78 (+9.61). VISE drops LogicVista and MMStar at every scale; VQS raises both, and all ten. Majority-vote methods barely move MMMU; VQS's +7.86 is about four times VISE's +1.75.
On the same 500 questions, VQS answers are 94.4% correct among retained items (keep rate 76%), against 76.4% for majority vote (keep 92.0%) and 82.2% for a model judge (keep 90.8%). Selecting parser targets per claim reaches 87.6 parse precision and 96.1 answer accuracy; scoring the whole parse once gets 85.0 and 94.0. Questions from an untrained parser push the solver to 58.0, below the 2B base of 59.22, because 13.2% of those answers are wrong. Label-free parser training lifts Structured-F1 by 5.0 on GQA, 6.6 on ChartQA, and 7.4 on AI2D-RST diagrams (75.2 to 82.6).
Dropping parser training cuts solver accuracy from 62.4 to 58.0; dropping the fact-checker cuts it to 60.9. Chart-difference templates gain +7.4, diagram connectivity +5.8; object existence is -0.2. Raising template families from 10 to 70 lifts the gain from 1.9 to 3.18. Mid-difficulty families score 62.4, easy 60.8, hard 61.1. Model-written programs match their question only 85.3% of the time and cost 1.0 solver point. Three cycles on the same images, resampling toward half-solved items, take the 2B gain from 3.18 to 3.41 to 3.84; knowledge and reasoning reach +5.94. Gemma3-12B, InternVL3-8B, and Llama-3.2-11B each rise on all ten benchmarks (+1.77, +1.51, +1.84), less than on Qwen3-VL.
The hole in self-evolution is a reward that comes from the model. VQS attaches a checkable reward to vision questions that have no gold label: parse, compute, then verify one claim at a time.
The practical case is a pile of unlabeled images and an open VLM you still want to push, especially on charts, diagrams, and multi-hop facts. Code is public. The cost is handwritten templates; families the library does not cover do not move. Object-existence items, which the base model already handles, gain almost nothing.
This is a change of verifier. On the same questions, vote and judge labels were wrong about one time in five; program-checked labels are wrong about 6% of the time.
There is no dedicated limitations section. Parser, checker, and solver share weights, so the checker can nod along with the parser's blind spots. An appendix grades 1,417 claims with five checkers: the 2B self-checker lets through 1.2% of 247 traps, but among claims it accepted that at least two other checkers rejected, diagram arrows are 26.7%, chart values 8.3%, and object existence 0%. An independent check needs a larger model, not a different family.
Human raters still mark 5.6% of VQS answers wrong; a bad parse makes the program compute the wrong key cleanly. Paraphrase is not checked, and four 8B cases swap the two years in a comparison. The vision tower never trains. Gains shrink with scale: RL changes 7.82% of benchmark answers at 2B and 4.93% at 8B, while training reward hits about 0.80 at all three sizes, so larger models learn the synthetic task and transfer less of it. Adding template families still raises the curve, so coverage is a ceiling. Training images come from the same pool as VISE; transfer beyond that pool is untested.