Remember-R1: Three Process Rewards That Curb Visual Forgetting in Long Multimodal Reasoning

Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning

Jianmin Chen, Jiaqi Tang, Wei Wei, Xiaogang Xu, Jiafei Wu, Zhe Liu, Qianzhou Wang, Yingying Yan, Botong Geng, Yuyang Xia, Lei Zhang, Qifeng Chen

cs.CV

2026-08-02

Long multimodal reasoning forgets the image. Remember-R1 adds three process rewards on the original trajectory without changing inference; MathVista +13.6 on Qwen2.5-VL 3B.

What problem this solves

Multimodal LLMs increasingly lean on long chain-of-thought to handle hard tasks, reasoning step by step before answering. As the chain grows, a subtle failure sets in: attention to the image decays as generation proceeds, and later tokens get driven more by the text the model has already produced than by the image. The paper calls this visual forgetting. A model counting geometric shapes can hallucinate a blue sphere that is not there and count from that phantom midway through its reasoning.

Two existing fixes each fall short. One re-injects the image or its features back into the context during inference; it recovers visual evidence but pays for it in extra compute and memory, and it fragments the reasoning. The other inserts visual-claim proxy interactions during training, but the supervision lands on those added proxies rather than on the model's own reasoning trajectory.

Remember-R1 takes a third path: leave inference untouched and apply process-level supervision directly on the trajectory the model itself generates.

Method

The framework trains with GRPO; the substance is in the reward design. Alongside a standard answer-correctness reward racc, it adds three process-level rewards, each watching a different facet of visual evidence.

Training data extends ViRL39K with annotations: Qwen-VL-Max extracts a set of visual keywords per image (object categories, colors, quantities, spatial phrases such as "leftmost bar"), and bounding boxes mark the regions that carry evidence for the question. The answer is used only during annotation and is never shown to the policy. After manual verification this yields 38,657 annotated samples.

The three rewards divide the labor:

• Visual vocabulary reward rvoc: encourages the reasoning to mention the annotated visual keywords. Matching is exact lowercase phrase match with no synonym expansion; each keyword is weighted by the position of its last occurrence, so mentions later in the response score higher. The intent is to keep the model citing visual facts deep into the chain, not just naming them up front.
• Visual memory reward rmem: takes final-layer attention weights, computes mean attention to visual tokens over an early window (μstart) and a late window (μend), and sets the reward to 1 + (μend − μstart). Attention that holds or rebounds at the end scores high, directly counteracting late-stage decay.
• Visual key-region reward rregion: maps annotated boxes to their visual tokens, computes the share of attention falling on those regions per step, and takes a time-weighted average with weight (t/T). Focusing on question-relevant regions later in generation earns more, guarding against attention that stays high but drifts to irrelevant corners.

The total reward is the sum of the four terms. The authors back the three-way decomposition with benchmark-level evidence: correct responses match more visual keywords on average (20.76 vs 12.50), show smaller early-to-late attention decline, and hold stronger attention on key regions.

Results

Evaluated on Qwen2.5-VL-3B and 7B across seven benchmarks: reasoning (MathVision, MathVista, LogicVista), general multimodal (MMVet, MMMB, MMStar), and perception (RealWorldQA).

Over the matching Qwen2.5-VL base, MathVista is the biggest gain: +13.6 on 3B and +7.5 on 7B. The 7B model reaches 72.37 on MMVet, ahead of methods built specifically to mitigate visual forgetting, DeepSketcher-7B (69.54) and TVC-7B (60.41).

Ablations show that removing any reward degrades results, and without rvoc the model drifts toward more abstract, text-driven reasoning. A GRPO variant trained only on racc reaches just 60.20 on MathVista versus 65.50 for the full method, evidence that answer-level supervision alone does not shape the process. Attention-curve analysis shows Remember-R1's attention to the image decays more slowly through the middle and late stages.

Why it matters

Two things matter for practitioners working on visual reasoning. First, inference is untouched: a trained model deploys as a normal model with no tax from re-injecting the image, so adoption is straightforward. Second, it turns visual forgetting from an observed phenomenon into an optimizable training signal, putting attention weights directly into the reward. The idea is clean and transfers to other long-chain tasks that need sustained grounding in external evidence (documents, charts, video).

The honest caveat: against other 7B methods that also target visual forgetting, Remember-R1's margin is thin (69.80 vs 69.10 on MathVista), closer to a steady improvement than a decisive win.

Limitations

The paper has no dedicated limitations section; these are implicit issues plus what stood out on reading:

• The whole signal rests on attention weights. The authors concede that Ω(t) "does not fully characterize model behavior"; high attention is not the same as genuinely relying on the right evidence, leaving room for reward hacking, where the model learns to dump attention onto visual tokens without truly depending on them.
• Visual keywords come from offline Qwen-VL-Max annotation plus manual verification, so the 38,657-sample scale and quality hinge on that pipeline. Matching is exact phrase with no synonym expansion, so a small wording change breaks the match, which is brittle.
• The benchmarks skew toward reasoning and general evaluation and omit the harder cases most likely to trigger forgetting, such as long documents and long video. RealWorldQA gains are roughly flat, so whether perception truly benefits is open.
• Only Qwen2.5-VL is tested as a base; replication on Qwen3-VL or InternVL is not shown.

Terms

• Visual forgetting: attention to image tokens decays as reasoning proceeds, with later tokens leaning on already-generated text instead of the image.
• Process-level reward: the reward is given not only on the final answer but at intermediate steps of the reasoning, scored by rule.
• GRPO: Group Relative Policy Optimization, an RL method that samples a group of responses per prompt and updates the policy on group-normalized advantages.
• racc: the answer-correctness reward, the most common reward type, judging only whether the final answer is right.
• Visual keywords: short lexical units extracted from the image (object categories, colors, quantities, spatial phrases) used to supervise whether the reasoning cites visible evidence.

Source

• arXiv 2608.01314v1
• PDF
• Hugging Face

What people are saying

• Remember-R1: Tackling 'Visual Forgetting' in Multimodal Long-Chain Reasoning 新智元

Related papers

• 26-way multi-hop video QA: Gemini 2.5 Pro still scores 51% in one pass
• Block-sparse featurizers recover 2–4D visual manifolds, beating SAE atoms on MDL
• HSImul3R cuts human-scene mesh penetration from 69.5% to 22.9% with a physics loop
• Short-Answer Hallucinations in MLLMs Start in Vision; ACFT Fixes Them on 0.9% of COCO
• LingBot-Map Hits 20 FPS on 10k-Frame Streams, Beating Offline 3D Models on Oxford Spires
• GRF-Recon hits KITTI 7.18 m ATE, matching calibrated SLAM without intrinsics

All paper explainers