Agent Training Data: More Tools, Harder to Generate
Shahules786 · x · 2026-07-14
This thread discusses how to generate post-training data that genuinely enhances agent capabilities. The core insight is that in enterprise-level, multi-turn long tasks, the number of tools expands exponentially. A single domain might have 100+ tools, and three domains easily exceed 300. Simply stuffing the tool list into the context degrades task performance.
The thread highlights three papers:
- PlanBench-XL: Constructs a tool environment closer to real-world scenarios, simulating agents working without seeing the full toolset;
- TMax: Focuses on synthesis and training for harder tasks;
- Autodata: Uses multi-model collaboration to determine if a task is feasible and trainable.
The author notes these papers are already influencing their internal practices: synthesizing tasks from tool graphs and combining multiple models of varying capabilities to assess trainability, with the ultimate goal of integrating these methods into a customizable, automated data pipeline.
Related event: PlanBench-XL Tackles Agent Tool Retrieval at Scale(3 posts)→
More from coding & agent
- IDA 9.5 ships Qualcomm Hexagon decompiler plus IDA MCP for AI-driven reverse engineering — OwariDa · 2026-10-27
- For OSS apps, proofs of concept — not PRs to one repo — may be the core primitive — dbreunig · 2026-10-27
- Hook everything into the Dot: a workaround to stretch ChatGPT subscription limits — teortaxesTex · 2026-10-27
- Jev launches people-search API for agents at $0.0089/lead, claiming 85% cheaper than Clay — alexcovo_eth · 2026-10-27
- Going through every compound engineering skill to decide where humans stay in the loop — kieranklaassen · 2026-10-27
- UCSB CS Prof. Xifeng Wang lands 3 grants worth $180K+ for practical agentic AI research — xwang_lk · 2026-10-27