Verus-Insight: Towards Explainable Artifact Detection in AI-Generated Videos
Zhuoran Zhang, Yang Shi, Qixun Wang, Tengfei Liu, Yuqi Tang, Xuanyu Zhu, Yizhuo Jia, Jingyun Hua, Shixuan Liu, Xianghua Ying, Haotian Wang, Yuanxing Zhang, Pengfei Wan, Wenjing Yang
cs.CV
2026-10-22
Three-stage training on Qwen3-VL-8B raises RewardData artifact Finding F1 from 3.38 to 35.42 and balanced accuracy from 48.81 to 94.79.
Synthetic video is now hard to reject by eye. Moderation, quality debugging, and provenance checks need the offending interval and region, alongside the real-or-fake label.
Detectors ask a multimodal model (an MLLM: it watches video and writes text) to name generation artifacts and then decide. Two failures recur. The model misses a small spatiotemporal glitch, or it treats ordinary light and jump cuts as artifacts. The real-or-fake call can also be right while the cited evidence is wrong or incomplete. In the barbershop clip of Figure 2, hair volume jumps with no plausible cause. One model never inspects that change. Another sees it and calls it styling. A third predicts fake and still misses the artifact. Clearing one stage does not clear the next.
Verus-Insight runs three stages on Qwen3-VL-8B-Instruct.
Inspection planning writes a checklist for that specific video. Each item names a property to examine and does not say whether the property fails. Evidence verification walks the items, marks each one anomaly or normal, and attaches a time span and a box when it can. Evidence aggregation may use only those checked observations. It does not add a fresh artifact claim on the way to the verdict.
The trajectories are built to block a shortcut. Annotators record each perceptible artifact's type, description, time, and region. That failure is rewritten as a neutral checklist item, and the original note stays as the verification target. A checklist made only of known failures would leak the answer, so the set is filled out with verified-normal items: candidate checks on objects, interactions, and timing are kept only after a separate check confirms the picture is normal. Pairs are shuffled so position cannot leak the label. Generated clips come from Kling-2.5, Veo 3, HunyuanVideo-1.5, LTX-2.3, and Wan2.2. Real clips come from NExT-QA, OneThinker-STVG, DiDeMo, and the web. The RewardData training split is mixed in.
Stage I warms up region tracking and checklist writing at a 1:1 ratio. Tracking takes an initial box and predicts boxes at queried timestamps. Stage II is supervised fine-tuning on the full trace, so the model itself emits the description, checklist, grounded checks, aggregation, and verdict. Stage III applies GRPO, a group-relative policy objective. Invalid format or a wrong real-or-fake verdict scores 0. A correct, valid verdict gets a base reward of λ, then extra credit: Finding F1 on fake videos, and credit on real videos only when the model names zero artifacts. The paper never states λ.
On the development set, warmup plus an explicit checklist is the best planning variant, at Plan F1 23.61, Finding F1 18.68, and balanced accuracy 86.11. For verification, region-tracking warmup plus explicit spatiotemporal grounding is best, at 20.28, 18.25, and 83.16.
Verus-Bench is sample-disjoint from train and dev. It targets small, local, brief artifacts, with frame-level human boxes and a manual quality pass.
Balanced accuracy (BA) averages recall on real and generated video. Plan F1 asks whether checklist items cover human-marked artifacts. Finding F1 asks whether those artifacts are recovered under one-to-one semantic matching. Unmatched predictions count as false positives, including claims on real video. Plan is scored on artifact-positive videos. Finding is scored on the full set.
| Benchmark | Metric | Qwen3-VL-8B | Verus-Insight |
| RewardData | BA | 48.81 | 94.79 |
| Verus-Bench | BA | 66.67 | 89.06 |
| Artifact-Bench T1 | BA | 50.80 | 66.60 |
| ViF-Bench | BA | 54.25 | 60.51 |
| RewardData | Finding F1 | 3.38 | 35.42 |
| Verus-Bench | Finding F1 | 7.56 | 15.97 |
| RewardData | Plan F1 | 18.67 | 31.33 |
| Verus-Bench | Plan F1 | 17.08 | 24.70 |
On RewardData, finding precision/recall moves from 1.87/18.13 to 33.66/37.36. Both ends move up. Finding F1 of 35.42 is the highest number in the table.
Video-level scores and artifact scores come apart. VideoVeritas posts 91.12 BA on RewardData with 8.47 Finding F1. Among open models, Verus-Insight's 94.79 and 35.42 lead that pair, and it also leads on Verus-Bench BA and on Artifact-Bench T1 and T3. Gemini 3.1 Pro still scores 69.60 on T1. T3 is exact-match accuracy: 8.29 for Verus-Insight, 4.86 for the backbone, 12.00 for Gemini 3.6 Flash. On ViF-Bench, Verus-Insight scores 60.51, behind Skyra at 63.81 and Gemini 3.6 Flash at 72.22. Artifact-Bench T2, which is plain accuracy, is 45.60, below VideoVeritas at 50.40 and below its own T1 BA of 66.60.
On RewardData, Gemini 3.6 Flash reaches Plan F1 33.13 and Finding F1 25.54, against 31.33 and 35.42. On Verus-Bench Finding F1, Gemini 3.6 Flash scores 27.63 and GPT-5.5 scores 23.53, both above 15.97. GPT-5.5 ties Verus-Insight on Verus-Bench BA at 89.06.
When a reference artifact is already covered by a matched checklist item, the chance it becomes a matched finding rises from 45.16% to 54.88% on Verus-Bench and from 30.48% to 75.58% on RewardData. The backbone has higher plan recall, so the new points come from verification. Strict artifact false-positive rate on real RewardData video is 1.20%, the lowest mark in Figure 5.
The pattern transfers. On the ViF-CoT-4K subset, Plan F1 is 35.27 and Finding F1 is 14.02, above VideoVeritas at 8.08 and BusterX++ at 5.34. Skyra is left out because of train overlap. On Verus-Bench, recall stays ahead of those specialist detectors for videos with three or more artifacts and across temporal and spatial size bins.
Ablations split classification from explanation. Dropping the Stage II explicit checklist raises BA to 90.63 on Verus-Bench and 96.07 on RewardData, from 89.06 and 94.79, while Plan/Finding F1 fall to 21.54/13.76 and 26.57/29.35. Dropping RL leaves Verus-Bench at Plan F1 20.99 and Finding F1 13.53. Dropping perception warmup or checklist warmup lowers all three metrics.
A high real-or-fake score can sit on thin evidence. VideoVeritas is the clear case: strong BA, weak Finding F1. Verus-Insight writes checklist, check, and aggregation as one trace, and aggregation cannot append an unchecked claim at the end.
The training recipe moves these numbers. Untuned Qwen3-VL-30B-A3B scores 65.10 BA on Verus-Bench and 52.70 on RewardData, both under the tuned 8B. With Finding F1 still around the mid-teens on held-out data, the practical slot is a draft in front of a human reviewer.
Absolute recovery stays low. Finding F1 is 15.97 on Verus-Bench and 14.02 on ViF-CoT-4K, so most human-marked artifacts are still missed. The jump from 3.38 to 35.42 on RewardData is the headline, and that split was part of training supervision. It is an in-distribution number.
The explicit checklist can hurt the metric many teams would ship. Removing it improves BA. GRPO also zeroes the whole reward when the real-or-fake call is wrong, so a careful artifact writeup on a misclassified clip earns nothing. Whether that gate throws away useful evidence is untested.
On Artifact-Bench, BA 66.60 and accuracy 45.60 differ by 21 points, with no account of the gap in the main text. Lambda, annotator agreement, and runtime are unreported. The appendix says coverage gains on RewardData are uneven across temporal and spatial bins. One-to-one semantic matching can inflate F1 if the matcher is loose, and match error is not broken out.
Training generators stop at Kling-2.5, Veo 3, HunyuanVideo-1.5, LTX-2.3, and Wan2.2. The author list includes the Kling team. The paper does not report a held-out slice for that generator alone, and later generators sit outside these tests.