ToolLoop Synthesizes Tool-Use Data in Three Feedback-Driven Stages, Hits 86.4% on BFCL

机器之心 · wechat · 2026-09-20

ToolLoop, a study completed at vivo AI Lab (accepted to EMNLP 2026 Main), replaces the one-shot 'generate then filter' pipeline for tool-call training data with a decomposed, feedback-driven process.

Method

• Candidate functions: semantic K-means clustering of function descriptions builds coherent tool sets for multi-tool selection and parallel calls.
• Three-stage generation: (1) sample the target function sequence as intermediate supervision; (2) reverse-generate the user question so intent matches the planned calls; (3) forward-generate structured calls with full parameters.
• Dynamic self-feedback: each stage combines LLM semantic checks, deterministic rules, and AST parsing; failures trigger stage-targeted regeneration (up to 3 retries), salvaging samples that naive pipelines would discard.

Results

• Fine-tuning Qwen3-4B-Instruct-2507 on 11,024 synthetic samples reaches 86.40% on BFCL single-turn, beating APIGen-4B (83.11%) and ToolMind-4B (83.53%); 72.1% on ACEBench.
• Ablations: no feedback 79.97%, final-only filtering 82.56%, full ToolLoop 86.40%.

Limitation: validation still relies on function definitions and model judgment rather than real API execution; multi-turn interaction is left for future work.

Original post →

More from Research

• Jitendra Malik: robots ignoring 3D structure are wasting a valuable signal — JitendraMalikCV · 2026-09-21
• Biological Millennium Problems Should Be Specific, Verifiable Datasets, Argues Poster — owl_posting · 2026-09-21
• Gzip, RE-PAIR Grammar Induction and the Case That Deep Learning Is Just the Cerebellum — ryunuck · 2026-09-21
• Encoder-style classification gets hot again: one multimodal BERT solved 1,000+ classification tasks — cwolferesearch · 2026-09-21
• Deep learning is brute force: activation functions can't encode epistemic structure, argues ryunuck — ryunuck · 2026-09-21
• ICLR's LLM reviewer: 1-2 useful points buried in three pages of nitpicking — Entrepreneur7962 · 2026-09-21

Research channel →