A 48M Parameter Specialist Model for Agent Tool Calling
Turbulent-Sky5396 · reddit · 2026-08-25
The author experimented with using a tiny 48M parameter model specifically for agent tool calling, replacing the task typically handled by large models. The core trick involves using a grammar compiled from schemas to emit JSON structure, while the model only answers a limited set of decision questions (e.g., to call or not, which tool), eliminating format errors and hallucinated parameters.
- Performance: On trained function catalogs, it outperforms the small baseline by 20+ points (86.3 vs 63.7); on unseen catalogs, it is roughly on par with the baseline and below frontier models.
- Cost: Total build cost was about $260. It is open source with weights included.
- Usage: A script is provided to specialize the model to your own API for about $56 in synthetic data.
Related event: 48M-Parameter Model Handles Agent Tool Calls(2 posts)→
More from coding & agent
- Microsoft Paper Proposes FORCE-Bench and Harness Architecture for Enterprise Agents — solyarisoftware · 2026-08-25
- Cursor Generates and Installs Mouse Wheel App Switcher Plugin Instantly — debreuil · 2026-08-25
- PyMC Labs: Speed Optimization Causes Agents to Skip Self-Checks — twiecki · 2026-08-25
- Rust MCP Memory Server with Provenance and Contradiction Handling — casualboy_10 · 2026-08-25
- Superset launches Design Mode: click elements to chat with any agent — ycombinator · 2026-08-25
- Trace2MCP: Compile Agent Traces into Verified MCP Workflows — War_Enterprise · 2026-08-25