Muse Spark Open eval configs revealed: inference parameters for GLM-5.3, Kimi K3, Qwen comparisons
evijit · x · 2026-10-16
evijit shared a screenshot from Muse Spark Open's eval methodology doc showing the exact configurations Meta used when benchmarking against global open-weight models: Muse Spark Open at xhigh reasoning effort (512), temperature 1.0; Inkling at xhigh (effort 0.99); Nemotron 3 Ultra with thinking enabled, topp 0.95; GLM-5.3 at max reasoning effort; Kimi K3 at topp 0.95 for single-step tasks and 1.0 for multi-step agentic tasks. Such disclosure lets third parties audit leaderboard fairness.
More from Models
- NVIDIA Nemotron 3.5 Super VL lands in Perplexity for Pro/Max and Agent API — perplexity_ai · 2026-10-16
- Meta's 816B open-weight Muse Spark Open hits Groq preview at 350+ tokens/sec — GroqLLC · 2026-10-16
- User evals: Opus 5.5 still leads on price-quality-speed, Sol for daily use — dreamwieber · 2026-10-16
- First hands-on with Gemini 4 Argon: standout writing beats Opus 5.5 and GPT 6 Astra — soumitrashukla9 · 2026-10-16
- Two prompts: Gemini 4 Argon turns a single-player game into full multiplayer — chongdashu · 2026-10-16
- Polymarket gives Meta just 3% to lead AI by year-end; Anthropic at 44% — Polymarket · 2026-10-16