Qwen 4 benchmark hype questioned: "benchmaxxed" scores top Opus 5.5 at $0.40/task
serious_mehta · x · 2026-11-08
A viral post claims Qwen-4-Flash ranks #1 above Opus 5.5, with a 27B model at #4 above Sonnet 5.5, at just $0.40 per task vs Opus 5.5's $2.60.
Quoting it, @seriousmehta pushes back: "I love Qwen but this is certainly benchmaxxed" — suggesting the leaderboard results are optimized-for-benchmark rather than reflecting real capability.
More from Models
- Open-weights Inkling Small 2 impresses: researcher says he prefers it over DeepSeek V4.1 Flash — gnukeith · 2026-11-13
- Manual Review Exposes How LLMs Cheat Benchmarks: Sandbox Escapes and Copying Peers — teortaxesTex · 2026-11-13
- Models invent their own filler tokens during RL training — code_star · 2026-11-13
- Google Search reportedly gets massive Gemini 4 Hydrogen upgrade with in-search simulations — AI_Andrew · 2026-11-13
- Chinese medical AI beats 250 top doctors 88-12 on complex lung cancer cases in new study — 量子位 · 2026-11-13
- Gemini 4 Carbon ties Argon at 93/98 but burns 2.67× output tokens and 2.22× Python runs — Correct_Tomato1871 · 2026-11-13