Benchmark: Tool Calling Performance of Qwen 35B-A3B Variants
OsmanthusBloom · reddit · 2026-08-26
The author benchmarked the tool-calling capabilities of Qwen3.6-35B-A3B and its fine-tunes. Using the tool-eval-bench 2.6.0 suite (88 tests in Hardmode) on a cluster of 32GB V100s, the results showed that Ornith 1.5 and Tiel-Coder (based on Ornith) were the top performers, scoring close to Qwen3.8-27B and significantly higher than Qwen3.6-27B. KAT Coder also slightly outperformed the original 35B-A3B. The tests utilized llama.cpp with Q4 quants across multiple runs, accounting for context pressure.
More from Models
- GLM-5.3 unprompted reasons about how its actions affect its future training, worries Apollo — MariusHobbhahn · 2026-10-15
- Gemini 4 argon vs Opus 5.5 hands-on: 3 min for $3 vs 25 min for $8 — zacharynado · 2026-10-15
- SentinelOne says OpenAI did 4-6 weeks of reverse-engineering in 8 hours — OpenAI · 2026-10-15
- Qwen3.8 27B scores 23.57% on word-based addition without reasoning, 167/169 with it — dl_weekly · 2026-10-15
- Hacker turns EmbeddingGemma 2 into 288x realtime STT and a fast TTS — kastnerkyle · 2026-10-15
- Claude Fable 5.5 reportedly in partners' hands, release eyed next week — lyraxana · 2026-10-15