SkillSeek: plain BM25 matches LLM-mediated agent skill retrieval at half the cost
StevensAGI · hf · 2026-10-01
Stevens AGI released SkillSeek, an open-source two-stage skill retriever tackling the bottleneck of selecting from 230,000+ aggregated agent skills.
- Prior art outsources skill selection to an LLM-mediated loop inside the agent, paying tokens on every task
- SkillSeek uses the standard IR recipe: a BGE-base bi-encoder first stage plus a small cross-encoder, exposed over MCP
- On the 89-task SkillsBench across a 4x11 grid, plain BM25 matches or beats the refined LLM loop of Liu et al. in three of four settings; the cross-encoder covers the fourth
- Per-trial spend drops from USD 51.30 to USD 27.54, within fifty cents of the no-skill baseline
- Conclusion: deterministic IR should be the default for agent skill retrieval, with LLM loops reserved for cases where it falls short
More from coding & agent
- Dev Burns Through Claude Weekly Quota in Under Two Days on Opus 4.5 Alone — yihui_indie · 2026-10-01
- Split coding workflow: Opus 5.5 xhigh for planning, Sol 6.1 high for the bulk of coding — haider1 · 2026-10-01
- FV expert: AI folks' view of formal verification is 15 years out of date — tianyin_xu · 2026-10-01
- a16z's Eric Jang: Codex Computer Control Beats Email APIs for Managing Inboxes — ericjang11 · 2026-10-01
- Why Agentic Inference Needs P/D Disaggregation: Decoding 300K-Token Codebase Latency — abhijithneil · 2026-10-01
- Meta's RankEvolve: multi-agent cross-checking lifts auto-research execution accuracy to 62.5% — _reachsumit · 2026-10-01