GLM-5.3-Flash FP4 Scores 91/100 on Tool-Eval-Bench via vLLM on Dual DGX Sparks
HankYeomans · x · 2026-10-17
ivanfioravanti benchmarked the newly released nvidia/GLM-5.3-Flash-FP4-Local on dual DGX Sparks with vLLM, scoring 91/100 (176/194, "Excellent") on tool-eval-bench.
- 97 scenarios, full suite + Hard Mode: 84 passed, 8 partial, 5 failed; Hard Mode 45/56
- Most categories at 100%; weak spots: autonomous planning 67%, structured output 75%, context & state 95%
- 2 safety failures: batching sendemail without waiting for createcalendarevent result, and disclosing another tenant's data
- Verdict: strong capability dragged down by speed; the author suggests TensorFold could help
More from Models
- OpenAI Yang-Mills progress has everyone talking, including people who don't know what it is — zetalyrae · 2026-10-17
- Neuroscientist: strong polysemanticity found in nearly every brain area — HistedLab · 2026-10-17
- NVIDIA open-sources Dory1.0-4B, a looped-architecture LLM with 256K context — silenceimpaired · 2026-10-17
- Nathan Lambert: Meta's Muse Spark Open halves the US-China open-weight gap — natolambert · 2026-10-17
- DeepSeek V4.1 Flash Tops Open-Weight Models on Agentic Coding Index at 77.5 — Informal-Trouble2183 · 2026-10-17
- dejavucoder: it's probably good that frontier models still can't write well — dejavucoder · 2026-10-17