1,152-call test: one parameter word swings tool-call accuracy from 53% to 79%
dbreunig · x · 2026-11-04
nicolaygerold tested 6 namings for a single argument of Amp's artifact tool across Opus 5.5, Sonnet 5.5, and GPT-6.1 Sol (1,152 calls total), measuring how often models picked the right artifact kind.
- Names alone: format: markdown | html got only 53% correct; kind: document | visual reached 79%
- The chart shows that adding a one-line note per value lifted all namings to 97–99% — the description matters more than the name
- Mentioning format/markdown caused models to draw ASCII diagrams instead of writing HTML
Takeaway: parameter naming has a huge impact, but a short paragraph of description nearly erases the difference.
More from coding & agent
- Dev proposes LLM-generated ultra-rare content to build anime-style MMORPGs — rickasaurus · 2026-11-04
- Dev builds Chrome extension to bring back the classic Twitter bird on X — joonasvirtanen · 2026-11-04
- Developer lets his app record 'wishes' and auto-spawn OpenCode background agents to fulfill them — SubtleGradient · 2026-11-04
- Post-train your model on your own agent harness with OpenHands, Harbor, SkyRL and Snowflake — StasBekman · 2026-11-04
- Google Antigravity redesign leaked with context usage and quota indicators — clmt · 2026-11-04
- Microsoft's 4B Mage-VL-Cowork Hits 66% on OSWorld-Verified, Near Human Level — microsoft · 2026-11-04