SAPO from Xiamen University and NTU ditches the separate critic in PPO-style agentic RL
heghbalz · x · 2026-10-30
Researchers from Xiamen University and NTU introduce SAPO, a compute- and memory-efficient PPO-style framework for agentic RL. It exploits the LLM's autoregressive causal structure: pre-action next-token logits read out V(s), post-action logits read out Q(s,a), so a single causal model learns policy and value jointly in one backward pass — no separate policy-scale critic and no same-task rollout groups. Under identical sampling budgets SAPO matches PPO and substantially outperforms GRPO with far lower resource use. Paper and code are public.
More from Research
- Perplexity's pplx-embed-v2-context ditches gold-chunk labels, sets SOTA on retrieval benchmarks — _reachsumit · 2026-10-30
- Gavin Crooks: Maybe it's time to write some 'last papers' as AI absorbs knowledge — MaxUnfried · 2026-10-30
- DoorDash's PALATE: GPT-style user embeddings lift ad revenue ~1% in production — _reachsumit · 2026-10-30
- A sequential decision framework unifies evaluation of LLM-guided search policies — _reachsumit · 2026-10-30
- Alibaba's PFA progressively focuses attention to speed long-sequence generative recommendation — _reachsumit · 2026-10-30
- RecRSI: ranking-scored short trials cut wasted retraining in LLM-driven recommender self-improvement — _reachsumit · 2026-10-30