Tool calling is still just a prompt: gateway lessons from Qwen3's chat template
Main-Fisherman-2075 · reddit · 2026-10-27
The author realized while building a gateway that tool calling is just ordinary text in the prompt: Qwen3's real chat template pastes tool JSON into <tools> tags in the system prompt, the model emits a <toolcall> block as plain tokens, and the server (vLLM, llama.cpp, hosted APIs) parses it into the toolcalls field; results return as <toolresponse>-wrapped user messages. Closed models work the same — Anthropic's docs confirm a hidden system prompt for tools (286 tokens on Claude Opus 5.5). Design implications for the tool loop: cap tool results at 64 KB with a [cut: N more bytes] note, return failed tools as Error: text instead of failing the request, and disable tools on the last round to force an answer (contrast: LiteLLM's proxy runs one round and returns).
More from coding & agent
- Cloudflare's Clairic: a System One extraction model returning typed values with confidence scores — omarsar0 · 2026-10-27
- Fanout: open-source tool tames parallel Claude Code and Codex sessions before merging — crumb_ona_breadstick · 2026-10-27
- Rust LVGL-style embedded UI library twine ships, built largely with Claude — Necessary-Pilot-3971 · 2026-10-27
- Share prompts, not code: a reusable prompt suite for evaluating small models (4B-36B) — yaroslavvb · 2026-10-27
- Maze Launches Maze Code: AI Agents That Investigate Dependency and Code Vulnerabilities — AndrewM_Webb · 2026-10-27
- rumi: a shared-doc workspace for orchestrating multiple coding agents at once — MuzafferMahi · 2026-10-27