New Analysis on MCP Tool Selection Bottlenecks
Inevitable_Fee1895 · reddit · 2026-07-13
This discussion focuses on why MCP tool selection degrades as the number of tools increases. The core conclusion is that retrieval-based pre-filtering only fixes part of the problem; the real bottleneck likely lies in the model's readout phase.
Key Points:
- Industry experience shows agent tool selection accuracy drops significantly when the tool count exceeds 20.
- A common fix uses RAG to retrieve a tool catalog shortlist for the model, which papers confirm significantly boosts accuracy and cuts prompt tokens.
- However, new research indicates that input-side fixes hit an early ceiling:
- In real BFCL failure cases, models mostly focus on the correct tool anyway.
- Input-side interventions like reordering or repeating tools fix only about 23% of failures.
- Readout-side interventions like attention bias or residual steering fix 59%–91% of failures.
- The author also introduces a label-free, gold tool-free selector using segmented attention as a confidence gate, improving function name accuracy on both BFCL and Seal-Tools.
The original post concludes that retrieval remains worthwhile for reducing costs and latency, but relying solely on retrieval and prompt optimization likely caps out here. The next step is exploring the readout layer, logit processors, constrained decoding, or external rescoring.
More from coding & agent
- Dev training a guard model to block prompt injection, tool and memory poisoning — gnukeith · 2026-10-27
- SF 'Code Engineering' event picks up where 1,000-person harness engineering meetup left off — LangChain · 2026-10-27
- Prompt turns Claude Code into an iPhone backup forensics tool exposing 30-day 'deleted' photo copies — alex_verem · 2026-10-27
- Prompt turns Claude Code into a forensics tool finding deleted iPhone photos in backups — alex_verem · 2026-10-27
- Phones won't be mobile IDEs but control planes for agent fleets, argues Yuchen Jin — Yuchenj_UW · 2026-10-27
- FrontierCS 2 launches on Kaggle: 48 tasks make agents beat published CS papers — MeganRisdal · 2026-10-27