DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection
Yuyang Hong, Jinhui Guo, Jiaqi Gu, Lubin Fan, Ruixiang Wang, Kun Ding, Yue Wu, Shiming Xiang, Jieping Ye
cs.CV
2026-08-31
DICS scores instruction data by image gain and response-instruction consistency. A 25% LLaVA-1.5 slice beats full data by 1.40%; 5.1M hits 94.52% of official InternVL3-8B-Instruct.
Visual instruction data is growing faster than the budget to train on all of it. Most selectors chase distributional diversity, sample influence, or heuristic filters. Few ask whether the image, the instruction, and the response inside one sample actually support each other.
Low visual-dependence items can be answered from language priors and tend to grow hallucinations. Responses that drift from the instruction break alignment. Text-only scores such as IFD have no visual channel. CLIP-Score mixes semantic relatedness with the harder requirement that the answer must look at the image.
A sample is an image I, instruction x, and response y. Two scores.
Visual Information Consistency (VIC) is the exponential of the weighted cross-entropy gap for predicting y with the image versus without it. Larger visual information gain scores higher. Response Information Consistency (RIC) inverts the problem: given the image, measure how much y helps reconstruct x. A response that actually answers the instruction raises RIC.
Tokens are POS-weighted so prepositions do not dominate. Multi-turn items average across turns. DIC is the minimum of VIC and RIC, so a high score on one axis cannot rescue a weak score on the other.
DICS then samples under a budget p. Below 50%, it builds a pool of the top 2p fraction and runs farthest-point sampling to keep diversity. At 50% it takes the top half. Above 50% it keeps the top (2p-1) core and FPS-fills the rest, which is a way to drop low-DIC redundancy. A 5% random warmup is used only for scoring and then discarded; final training starts from the original pretrained checkpoint. Text-only items get DIC 1.0.
LLaVA-1.5-7B on a 25% (166K) slice of LLaVA-1.5-665K:
| Method | Relative to full |
| Full 665K | 100% |
| Random | 96.69% |
| PRISM | 97.66% |
| CLIP-Score | 98.66% |
| DICS | 101.40% |
DocVQA is 24.64 versus 23.82 full; POPE is 84.91 versus 84.49. At 50% the relative score peaks at 103.15%, then falls as low-DIC items re-enter, still ahead of other selectors.
On Vision-FLAN, 25% reaches 99.91% of full data. Cross-architecture transfer holds: data scored by LLaVA-7B trains Qwen2-VL-7B to 101.25% relative, and trains LLaVA-13B to 101.79%.
On InternVL3-8B with the in-house DICS-6M corpus, 25% (1.5M) is 101.47% of the 6M full run; 85% (5.1M) is 103.10%. Official Instruct used 21.7M and sits at 109.08% of that same 6M baseline, so 103.10% is 94.52% of the official number.
Ablations: VIC-only 101.02%, RIC-only 97.60%. Sum or product fusion lands near 98%. Dropping POS weights falls from 101.40% to 100.25%. Diversity-only FPS is 97.02%. Scoring is forward-pass only; 665K takes about 12 hours on 8 A100s.
More visual instruction data is not automatically better. Internally inconsistent items drag alignment and feed hallucinations. DICS is a scorer that does not call an external GPT judge, scales linearly into the millions, and transfers from a small selector to larger or different backbones. For teams paying for a LLaVA or InternVL instruction stage, beating full data at 25% is a budget they can actually run.
This is solid data selection, not a new VLM architecture.
The authors restrict the metric to image-text; video and audio are out. Optimal subset size is not chosen automatically, and low-DIC items are dropped rather than repaired. Scoring depends on the base model's loss, and the 5% warmup is a mild circularity. The 94.52% comparison to official InternVL3-8B-Instruct is not a matched recipe: 21.7M data mix and training schedule differ, so treat it as a scale reference. LLaVA runs use LoRA for one epoch; absolute scores such as MMMU 36.11 remain far from modern specialist models. The comparison that matters is subset versus full under the same protocol. Code is at cqu-student/DICS.