ThunderAgent (ICML Spotlight): Nearly 2x Throughput for Multi-Turn Agent RL
simran_s_arora · x · 2026-07-30
Proposed by Together AI, ThunderAgent tackles inference scheduling bottlenecks in multi-turn Agent reinforcement learning. It has been accepted as an ICML 2026 Spotlight paper.
- Core Optimization: Introduces program-aware routing at the scheduler level to address severe KV cache thrashing during agentic inference.
- Performance: Delivers 1.94x rollout throughput and 1.39x full-step throughput in matched synchronous GRPO runs. Achieves 2.5x higher single-node throughput and 10x lower P50 latency under high concurrency.
- Integration: The solution has been merged into the verl-recipe open-source project as a Dynamo recipe, enabling efficient multi-turn Agent RL training.
Related event: Together AI Unveils ThunderAgent, Doubling Agent Inference Throughput(7 posts)→
More from coding & agent
- 'LLM code is sloppy' rebutted with real ffmpeg code: identical if/else branches in a top library — kingslyj · 2026-10-16
- Building an acceptance harness for AI spreadsheet edits: allowlists, recalc checks, idempotency — ukdicha-modakkk · 2026-10-16
- Agent actions need checks outside the model: NVIDIA demos lean on guardrails amid wave of real agent incidents — Creamy-And-Crowded · 2026-10-16
- Vitalik builds "paranoid mode" agent wrapper with Tor, zkAPI and local prompt rewriting — DavideCrapis · 2026-10-16
- Agent fine-tunes Qwen3.5-0.8B with synthetic data: 4.7%→76.3% accuracy, 11.9× faster — ben_burtenshaw · 2026-10-16
- Beam-501B agent demo: solo beats 5-agent team on ProgramBench rebuild task — ben_burtenshaw · 2026-10-16