DR-Rubric: Xiaohongshu & Fudan turn deep research into RL rubric rewards, accepted at NeurIPS
机器之心 · wechat · 2026-10-19
Xiaohongshu and Fudan University propose DR-Rubric (accepted at NeurIPS 2026): the model first runs multi-turn deep research to gather evidence, then distills it into atomic, checkable rubric constraints used as GRPO rewards for open-ended RL tasks.
Key results
- Only 1000 RL samples per round beat baselines by 1.3–3.8 points across six benchmarks (ResearchQA, DeepResearchBench, LocalSearchBench, GPQA, MMLU-Pro, MMLU)
- Benchmark-provided rubrics are not the training ceiling: GPT-5, Gemini, and bootstrapped rubrics can outperform them, with complementary capability profiles
- Bootstrapping is non-monotonic: BS-2 dips before BS-3 recovers; rubric structure diverges sharply across rounds (word-level Jaccard 0.13–0.15)
- Over-bootstrapping limit: on Qwen3-30B-A3B, reward polarization crossed the 0.15 threshold one iteration before collapse — internal rewards rose from 0.24 to 0.66 while true task scores fell from 51.3 to 22.7
Takeaway: reward quality matters more than data volume; bootstrapped rubric generation needs stopping rules or external recalibration. Paper: arxiv.org/abs/2606.01091; code on GitHub.
More from Research
- Domingos Cites New Result Claiming RL Is 'Basically Useless' vs Supervised Learning — pmddomingos · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23