HYSET scores tool sets instead of single tools, lifting agent task success to 69.9%
DeepLearningAI · x · 2026-10-16
DeepLearning.AI's The Batch covers HYSET, a method from Shanghai Jiao Tong University and Hong Kong Polytechnic researchers that selects collections of tools for LLM agents as a set rather than scoring each tool independently.
Key insight: a tool's utility depends on what other tools are available—for multi-step queries (flights + hotel + weather + currency), naive relevance ranking floods the list with flight APIs.
How it works: a small trainable scorer takes query embeddings from a frozen Qwen2.5-1.5B encoder, combining query-set alignment (relevance) and set-level scoring (tool compatibility), trained on ToolBench (14K tools, 200K instructions) with LLM-as-Judge reward-weighted self-training.
Results: Recall@5 improves from 81.4% to 88.6%; tasks resolved from 61.8% to 69.9%.
More from coding & agent
- Solo-built FrankenGraphDB beats $14M-funded Memgraph on several graph queries in benchmarks — doodlestein · 2026-10-16
- raylib 6.2 ships with 300+ improvements and 100 new contributors, 20 new API functions — jsuarez · 2026-10-16
- "Vibecoding" is as misleading a label as "AI" and "SI," dev complains — eschadiol · 2026-10-16
- Dev runs Pstack as a Grok bot to audit his own pipeline — it surfaces a pile of hidden bugs — alexcovo_eth · 2026-10-16
- Doc2Harness: New benchmark finds best coding agent passes only 53% across 329 cases — _akhaliq · 2026-10-16
- Perplexity open-sources SPACE, the Firecracker microVM sandbox behind Perplexity Computer — perplexity_ai · 2026-10-16