New paper compares 11 rewards for training budget-aware multi-turn agents
adam_golinski · x · 2026-10-22
A new arXiv paper (Lamb, Golinski, Kirchhof et al.) studies whether multi-turn LLM agents can learn to allocate inference budgets across tokens, API calls, steps, and turns. The team adapted existing length penalties—noting every lab (DeepSeek, Kimi, ...) uses a different one—and compared 11 reward objectives in a shared pipeline, training Qwen3.5 4B/9B on AppWorld and a policy on BabyAI. Overspending-penalized rewards substantially improve within-budget success and natural compliance, generalize to unseen budgets, and transfer zero-shot to a shifted distribution and new benchmark, while objectives trade off success against compliance.
More from coding & agent
- Scrawl: open-source Mac tool turns screen markup and voice into agent feedback, 680 stars — aronchick · 2026-10-22
- Where should agent autonomy end? The gray zone beyond blocking dangerous APIs — Innowise_ · 2026-10-22
- PR pre-review pipeline flags issues in 30s for one cent per review — please-dont-deploy · 2026-10-22
- Databricks to offer managed agent memory; how are devs handling it? — WorldOfUmbro · 2026-10-22
- Longman Dictionary MCP Server Serves Structured Definitions to AI Agents — modelcontextprotocol · 2026-10-22
- Law professor rethinks Microsoft Copilot after hands-on work inside a major law firm — jkubicki · 2026-10-22