Salesforce's Critical-State RL Gains ~14 Points on BFCL v4 by Training Only the Pivotal Tool Call
dair_ai · x · 2026-09-28
A Salesforce AI Research paper on RL for multi-turn tool use argues you should train only the call where the action changes the outcome, not spread reward across the trajectory: when reward depends on later turns, most variation comes from downstream noise. Critical-State RL uses nested sampling to isolate the reward variation caused by the current action, then trains just that call with contextual-bandit updates. On BFCL v4 missing-function tasks this adds 14 points, while training other candidate turns leaves accuracy flat or lower.
More from coding & agent
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16
- lemire: AI's real impact on coding is expanding the engineer's 'cognitive radius', not speed — lemire · 2026-11-16