ToolLoop Synthesizes Tool-Use Data in Three Feedback-Driven Stages, Hits 86.4% on BFCL
机器之心 · wechat · 2026-09-20
ToolLoop, a study completed at vivo AI Lab (accepted to EMNLP 2026 Main), replaces the one-shot 'generate then filter' pipeline for tool-call training data with a decomposed, feedback-driven process.
Method
- Candidate functions: semantic K-means clustering of function descriptions builds coherent tool sets for multi-tool selection and parallel calls.
- Three-stage generation: (1) sample the target function sequence as intermediate supervision; (2) reverse-generate the user question so intent matches the planned calls; (3) forward-generate structured calls with full parameters.
- Dynamic self-feedback: each stage combines LLM semantic checks, deterministic rules, and AST parsing; failures trigger stage-targeted regeneration (up to 3 retries), salvaging samples that naive pipelines would discard.
Results
- Fine-tuning Qwen3-4B-Instruct-2507 on 11,024 synthetic samples reaches 86.40% on BFCL single-turn, beating APIGen-4B (83.11%) and ToolMind-4B (83.53%); 72.1% on ACEBench.
- Ablations: no feedback 79.97%, final-only filtering 82.56%, full ToolLoop 86.40%.
Limitation: validation still relies on function definitions and model judgment rather than real API execution; multi-turn interaction is left for future work.
More from Research
- minpts-check: one-run predictor tells if your HDBSCAN clusters survive min_samples changes — leland_mcinnes · 2026-11-12
- Architecture study sparks monitoring concerns over a multi-hop planning spike — ChowdhuryNeil · 2026-11-12
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12