Ant Group's Ling team cuts agent eval batch time by up to 65% with smarter scheduling
dair_ai · x · 2026-11-17
dairai highlights a practical paper from Ant Group's Ling team, "Reducing Batch Makespan for LLM Agent Workloads through Inference–Environment Coordination."
- Agent tasks alternate between model inference and environment steps, so GPUs and sandboxes wait on each other; the longest trajectories determine when the whole batch finishes
- Key techniques: Chain-Advancing Action Scheduling (prioritize long chains by observed progress), trajectory-aware routing that keeps trajectories on workers holding their KV cache, and decoupling sample admission from environment execution when slots are limited
- A workload-based resource model guides GPU memory allocation between KV cache and recurrent state, with a reference sample concurrency
- On BFCL, SWE-bench Multilingual, WideSearch, and MCP-Atlas, batch completion time drops 10.4%–65.0% with unchanged eval scores; methods now run in Ling-3.x training and evaluation
- Controlled replay (real inference + recorded environment responses) enables per-optimization ablations
Takeaway: if your eval runs end with a few long trajectories while other workers idle, fix scheduling before buying more GPUs.
More from coding & agent
- Merge Ships 19 New Connectors for Its Agent Handler, From Terraform to Stripe — shensi · 2026-11-17
- Anthropic and Ode devs on mining the tacit knowledge workers leave out when teaching AI — every · 2026-11-17
- Rebuilding a $1.5M Medical AI Pipeline with LangGraph: Six Agents from Raw Data to Explainable Predictions — MaryamMiradi · 2026-11-17
- OSS maintainer uses Devin AI to cut duplicate docs builds from CI pipeline — DanielLockyer · 2026-11-17
- AutoRL: Multi-Agent System Automates RL Training, Lifts Qwen3 AIME from 46.7% to 66.7% — omarsar0 · 2026-11-17
- 9 Hermes Agent Plugins Worth a Weekend, Picked From 2,300+ — Teknium · 2026-11-17