ByteDance's CoRT: Token-Level Credit Assignment for GRPO Optimization
ByteDance · hf · 2026-07-30
ByteDance introduces CoRT (Counterfactual Replay), a method to optimize Rubric-based GRPO reinforcement learning pipelines.
- The Problem: In GRPO-style pipelines, structured judgments are reduced to a scalar reward broadcast uniformly across all generated tokens, lacking a mechanism for within-response credit allocation.
- The Method: CoRT uses counterfactual replay to rescore the same sampled response under both rubric-conditioned and criteria-free prompts. It uses tokenwise log-likelihood contrasts as a proxy to redistribute the GRPO advantage across tokens.
- Results: Without introducing an auxiliary scorer, CoRT outperforms standard response-level GRPO in the vast majority of comparisons with an average gain of 4.4 percentage points, retaining the simplicity and stability of GRPO.
Related event: ByteDance Proposes CoRT to Optimize LLM RL Credit Assignment(2 posts)→
More from Research
- Transforming quadruped robot runs on real hardware: per-pose RL policies in Isaac switched on morph — _Stocko_ · 2026-10-16
- KinoGen synthesizes robot loco-manipulation reference motions directly via kinematically grounded video generation — ChongZzZhang · 2026-10-16
- Science Robotics paper introduces F³, predictive event-based vision beyond optical flow — CSProfKGD · 2026-10-16
- VideoGen-Agent: An 8B Multimodal Agent Turns Video Generation Into a Tool-Use Loop — jiqizhixin · 2026-10-16
- Comprehension Audits: New Paper Proposes a Gate for AI-Written Code at Frontier Labs — ronbodkin · 2026-10-16
- Not Boring #215: Cammarata's Clean Networks Claim, Tesla's Flying Roadster, Tau Cut 73% — Not Boring (Packy McCormick) · 2026-10-16