Xiaohongshu's PACT unifies credit assignment and lifts SWE-bench to 67.4%
小红书技术REDtech · wechat · 2026-09-29
Xiaohongshu's AllSpark team published PACT (Policy-Aligned Critic Training), a mathematically grounded take on credit assignment for LLM agent RL training.
Theory
- Three regularity conditions — completeness, prefix consistency, and neutrality — are shown to pin down a unique credit assignment, formally the stepwise change in the conditional expectation of final reward.
- This unifies existing methods: OPD's teacher acts as an implicit Critic; RLOO's response-level advantage yields the same expected gradient as the unique credit; GAE's intermediate Critic errors can corrupt advantage estimates, removable with λ=1.
Algorithm
- Replaces MSE with BCE for regressing bounded values — faster convergence and lower error.
- Updates the actor first, then aligns the critic to the new policy via importance-sampling correction, reusing existing trajectories.
Results
- Qwen3.6-35B-A3B trained on OpenSWE reaches 67.4% on SWE-bench Verified, +2.0 over GRPO and +3.8 over SAO.
- Qwen3.5-4B averages 72.87% across four math benchmarks, +8.80 over GRPO; ablations show removing critic IS correction costs 5.13 points.
Paper: arxiv.org/abs/2609.26355; code open-sourced on GitHub.
More from Research
- Reddit asks: why no folding@home-style crowdsourced open training dataset? — a300a300 · 2026-10-24
- AI paper reproduction failures trace mostly to the papers, not the AI agents — RexDouglass · 2026-10-24
- Agents reproduce most published findings: ~80-90% correct signs, >70% within CI — RexDouglass · 2026-10-24
- Lafa: 8B flow language model hits 2.54x AR throughput with under 30B training tokens — LucaAmb · 2026-10-24
- Explainer: how the DFT exactly represents any finite sequence as complex oscillations — techNmak · 2026-10-24
- Fudan's F2G-Pose estimates 6D pose, size and shape from one RGB-D image at 28 FPS — joemeno · 2026-10-24