SIMIT: vision-language models self-improve at test time by imagining their own practice data
dtudk · hf · 2026-11-03
SIMIT, from DTU and University of Copenhagen, is a test-time self-improvement framework where a vision-language model creates its own query-specific training data before answering an unlabeled test query—no annotations or external models needed. Unlike longer reasoning or majority voting (model unchanged) or reward-based updates that can reinforce mistakes, SIMIT 'imagines' similar problems whose solutions it controls: it synthesizes diverse (question, answer, image) triplets as practice examples, combining native image generation with an extensible skill library for structured visuals like documents and charts, with multi-stage verification checking that images support their assigned answers.
More from Research
- They reverse-engineered a $1B-funded megakernel demo from video frames and open-sourced it — erikphoel · 2026-11-03
- NVIDIA, Tsinghua, CMU, Berkeley and Princeton launch StreamEval, first real-time interactive streaming video benchmark — JaynitMakwana · 2026-11-03
- MIT: AI edits 0.016% of another model's weights, lifting MATH-500 from 7.9% to 48.1% — rohanpaul_ai · 2026-11-03
- Schmidhuber: AI weight editing is 'old hat' from his 1990s fast weight programmers — SchmidhuberAI · 2026-11-03
- UCL's TAL lets LLM agents learn from tutorial videos with no parameter updates, +113% on VidLearn — UniversityCollegeLondon · 2026-11-03
- Face quality scores can backfire: higher quality often signals a morphing attack — FraunhoferIGD · 2026-11-03