Multi-effort RL: set cost penalty to the Pareto curve slope to lift the whole frontier
carlesgelada · x · 2026-09-12
Carles Gelada details a multi-effort RL reward design aimed at pushing up the entire Pareto frontier: reward R = S − λe·C, where S is solve/fail, C the rollout cost, and λe an effort-specific penalty. The key result: the correct λe equals the slope of the base model's Pareto curve at each effort level, approximated once and held constant during RL. This works because λe sets the slope of iso-reward lines; only when those lines are tangent to the Pareto frontier does improving reward guarantee frontier improvement.
Related event: RL Reward Design: Choosing λ_e to Lift the Whole Pareto Frontier(3 posts)→
More from Research
- ParetoRSI-Gym benchmarks whether AI agents can recursively self-improve LLMs across domains — Haojin Wang · 2026-10-28
- Stanford's Ψ-0.6 is a unified vision-language world model with adaptive reasoning streams — Stanford · 2026-10-28
- Shanghai AI Lab's OpenESM shows energy-steered models can scale as foundation learners — Shanghai-AI-Laboratory · 2026-10-28
- CellTrack-Bench tests VLMs on microscopic cell tracking, exposing reasoning gaps — biohub · 2026-10-28
- Hybrid LLM agent scores 83.2%, beats nine VLA policies on new SO-101 robot arm benchmark — Truman Hickok · 2026-10-28
- DeepMind researcher: all of mathematics will soon be formalizable, with agents hunting proofs — ChrSzegedy · 2026-10-28