ChronoVision: Temporal Reasoning via Latent State Reconstruction
Yifan Shen, Jian Xu, Boyi Li, Yuner Zhang, Tianjiao Yu, Bingxuan Li, Houze Yang, Rushi Wang, Xu Cao
cs.CV
2026-08-06
ChronoVision predicts the final frame's latent representation before ordering shuffled video frames, reaching 74.8% on its Vbvr-VQA benchmark and beating Claude Opus 4.6.
Multimodal LLMs recognize static images well but collapse on tasks that need multi-step simulation, such as where a rotated object lands or how a ball bounces after five rebounds. The authors pin the bottleneck on language: chain-of-thought runs in text, and continuous physical motion cannot be described in words without losing spatial detail.
They also fault the evaluations. Multiple-choice formats let models exploit ambiguous option text and guess right without looking at the image, and picking a single final answer never tests whether the model tracks a temporal sequence.
So they do two things: build a benchmark that closes the linguistic shortcut, and train a model that internally reconstructs the final visual state.
ChronoVision is built on Qwen3.5-9B and trained in two stages.
The supervised stage adds two modules. A Reconstructive Visual Head forces the model to predict the latent representation of the final frame from scattered cues in the shuffled candidates, supervised by mean squared error. This internalizes what the outcome looks like instead of just emitting a ranking. A Region of Interest Attention Locating module concentrates attention at an intermediate layer onto the region where the transformation happens, using an attention-condensation loss to push weights into the annotated bounding box and filter background noise.
The second stage is GRPO-based reinforcement learning with what the authors call implicit process grounding. The reward has three parts: an outcome reward for exact-match ordering, a latent-grounding reward measuring cosine similarity between each reasoning step and candidate visual features, and an unsupervised focus reward using Shannon entropy of self-attention to prevent collapse. All are bounded in [0,1] and need no human labels.
The accompanying Vbvr-VQA benchmark turns video reasoning into strict image ordering: given an initial frame and six shuffled candidates, the model must output the correct chronological sequence. There is no room to answer in words, so textual shortcuts disappear.
On Vbvr-VQA ChronoVision scores 73.2% overall, 74.8% in-domain, 71.6% out-of-domain.
| Model | Overall |
| Qwen3.5-9B (base) | 14.2 |
| GPT-5.4 | 33.8 |
| GPT o3 | 46.0 |
| Qwen3.5-397B | 49.4 |
| Claude Opus 4.6 | 55.8 |
| ChronoVision | 73.2 |
On the cross-domain IntPhys2 benchmark (ordering real physical events) it reaches 55.0%, up 6.5 points over the 48.5% base, and 11.5 points on the easy subset. Two real video benchmarks score 45.89 (Video-Holmes) and 74.7 (LongVideo-Reason). In ablation, the reconstructive head alone adds about 3.2 points, the ROI module 1.2, and the RL stage another 3.2; all three reward terms are necessary, with the outcome reward the most critical.
General vision-language benchmarks barely move: MMMU goes from 78.4 to 78.8. This is a specialized add-on, not a general intelligence gain.
Turning the cognitive-science idea of mental simulation into a training objective, and replacing language with latent reconstruction, is a clean idea. The ROI attention and process rewards both stay focused on visual evidence.
Keep one fact in view: the 74.8 number comes mainly from an author-built benchmark, and the frontier models are evaluated zero-shot on a format they have never seen, while ChronoVision is trained specifically for it. Part of the gap reflects format unfamiliarity, not pure capability. The flat general benchmarks confirm this is a narrow enhancement.
The biggest issue is evaluation self-cycling. Vbvr-VQA is the authors' own benchmark and task format, so a large lead for the home team needs third-party replication to hold up. IntPhys2 is an existing benchmark and more convincing, but the gain there (6.5 points) is far smaller than the lead on the in-house benchmark (about 18 points), and is closer to the real capability delta.
General vision-language skill does not improve, so the method is a special-purpose add-on for temporal ordering; transfer to other visual-cognition tasks is untested. The base is Qwen3.5-9B, and the paper does not report training-data scale or compute cost.