27-question self-built eval: Nemotron 3.5 Super beats Inkling 2 Mini and Muse Glimmer across the board
appakaradi · reddit · 2026-11-21
A community three-way eval on 27 identical questions ranks Nemotron 3.5 Super (120B-A12B) the clear winner: fastest median latency (51.3s), 2.4x fewer tool calls (9.1 avg), fewest failures (17), zero repeated calls, and the strongest tables/depth. Inkling 2 Mini (26B-A5.4B looped) beats Muse Glimmer 30B on structure and loop avoidance but is 5x slower (316s, no radix cache), with 19 wrong table/column guesses vs Muse's 3 and only 41% valid citations vs Muse's 67%. Neither approaches Nemotron.
More from Models
- Google appears to reset Ultra weekly limits for all Antigravity users — novice_invader · 2026-11-21
- Claims surface: Navier-Stokes and Riemann proofs via LEAN, by OpenAI and Claude — BasketVolleyball11 · 2026-11-21
- Yacine: 'It might be over for OpenAI — and the killer isn't Claude, it's Gemini' — yacineMTB · 2026-11-21
- InferenceBench ranks agents on LLM serving speedup: Claude Opus 5.5 tops at 12.08x, agents lose to simple hyperparameter search — maksym_andr · 2026-11-21
- Inkling-2-Mini quants are a trap: 17GB weights overflow 48GB VRAM at long context — EmilPi · 2026-11-21
- Cloudflare launches multimodal model clairic-flash, trending on Hugging Face — Cloudflare · 2026-11-21