Cursor & UC Berkeley's Targeted Guidance teaches RL new behaviors without degrading baselines
burny_tech · x · 2026-11-05
Cursor and UC Berkeley researchers (Kevin Frans, Alexander Rush, Sergey Levine, Pieter Abbeel, Ashvin Nair) released "Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning," #1 trending on alphaXiv.
Problem: naive on-policy RL only reinforces behaviors the base model already exhibits; when correct behavior never appears in samples, rewards say "failed" but not "how."
Method (Targeted Guidance):
- Use per-turn textual feedback showing the model how to correct specific mistakes
- Self-distillation: train the model to imitate its own feedback-guided predictions, but only on failed turns
- Continue standard RL alongside, preserving existing capabilities; design choices analyzed for stable entropy and less oscillation
Results: on Qwen3-Coder-30B in a multi-turn SWE environment, Markdown compliance jumps 0% → 85% while coding success stays at 33%, matching the RL baseline — new tool-use behaviors learned without capability regression.
More from coding & agent
- Voice prompting in Antigravity + Gemini 4: rambling your creative decision tree to the model — alexanderchen · 2026-11-05
- Voice prompting in practice: one long spoken prompt drives a full dev iteration — alexanderchen · 2026-11-05
- DrawSimple for Mac ships with an MCP server so agents can draw editable mockups — JordanRunsForFun · 2026-11-05
- Gradium and SurrealDB demo a fully on-device research agent with memory and streaming TTS — mattturck · 2026-11-05
- webIQ demo shows the evidence object returned to models, stressing freshness signals — VeryWellVersed · 2026-11-05
- 10 open-source GitHub repos for voice AI: cloning, TTS, and phone-call agents — victor_explore · 2026-11-05