Muse Spark 1.3 looks benchmaxxed: real-world tests far below its benchmark scores
Swimming_Gain_4989 · reddit · 2026-09-05
A Reddit user tested Muse Spark 1.3 via both OpenRouter and Opencode Zen, with different system prompts and outside a harness, and found it consistently underperforming — suspecting the model is overfit to benchmarks. It was passable at Godot debugging and documentation lookups (roughly DeepSeek v4 flash level), but performed like GPT-4 mini at tutoring upper-level discrete math, unable to sustain back-and-forth conversation and often giving incomplete or incorrect explanations.
Related event: Users Report Real-World Results Contradict AA Index Rankings(5 posts)→
More from Models
- Opus 5.5 tops OpenRouter spending in 20 of 29 task categories — mmurph · 2026-10-21
- Meta team builds open-source 30B LLM from scratch in months, hails 'second era' — TimDarcet · 2026-10-21
- DFlash drafter for Muse Glimmer-30B: 5 layers, blockwise 16-token spec decoding — TimDarcet · 2026-10-21
- EvoPress quantization shrinks 30B model from 56GB to 19GB at near-bf16 accuracy — TimDarcet · 2026-10-21
- Radar chart: Muse Glimmer-30B benchmarked against Gemma4-31B and Qwen3.6-27B — TimDarcet · 2026-10-21
- Privacy evals: Muse Glimmer-30B leaks less than Qwen3.6-27B on CIMemories — TimDarcet · 2026-10-21