ReMaRk Cuts GRPO Responder Noise in LLM Recommenders, Reaching Parity With 25–53% of Training Steps
_reachsumit · x · 2026-10-14
A new arXiv paper argues that Rank-GRPO, the SOTA recipe for RL-training LLM recommenders, picks only one reply per training example—so 85% of its advantage variance reflects which reply was sampled rather than list quality.
The proposed ReMaRk scores each generated list once against the union of all replies' titles and samples each request equally, making RL training cheaper:
- Across 7 backbones and 3 datasets, ReMaRk matches the validation recall of 2,000 per-reply RL steps using only 25–53% of the steps
- Trained until neither method improves, it is never significantly behind Rank-GRPO and beats it on GPU-hours in 4 of 7 pairs
- The same joint reward works on ProtoQA (100 answers per question), suggesting a simple rule: with multiple valid human answers, reward them jointly
More from Research
- Tufts and Santa Fe Institute paper models LLM adoption as a 'cognitive virus' — alex_verem · 2026-10-14
- 'LLMs as a Cognitive Virus': paper finds tipping point to population-wide AI dependence — alex_verem · 2026-10-14
- PyVBBC 1.5 Released After 3 Years, Bringing Major Upgrades to Bayesian Model Inference — twiecki · 2026-10-14
- AgentHorizon: 1,373-task benchmark exposes unreliable LLM judges for computer-use agents — rohanpaul_ai · 2026-10-14
- ServiceNow and McGill's AgentHorizon benchmark shows LLM judges miss subtle agent failures — rohanpaul_ai · 2026-10-14
- Cambridge's van der Schaar to keynote on AI and oncology at Momentum 2026 — MihaelaVDS · 2026-10-14