Redditor calls benchmark scores meaningless: 1.6T-param DeepSeek only 1 point ahead of 27B Qwen
Nerfariox · reddit · 2026-08-18
A Reddit user argues benchmark scores have become meaningless, pointing to Artificial Analysis' Intelligence Index: DeepSeek V4 Pro 0813 (1600B-A49B, 1.6T params) is only 1 point ahead of Qwen3.8 27B despite 60x the parameters. He also notes people cite the index to claim Qwen3.8 27B beats Gemma 4 31B, while his own day-to-day tests on puzzles and C++/Java refactoring consistently favor Gemma 4 31B.
More from Models
- Prediction: frontier AI revenue growth could slow even as AI usage keeps rising — yi111 · 2026-10-16
- Meta teased 816B-param open-weight Muse Spark Open with 1M context, says meme thread — alexandr_wang · 2026-10-16
- Nolla Health Post-Trains Nemotron 3.5 VL: Visual Diagnosis Jumps from 35% to 66.8% — LoganGrasby · 2026-10-16
- Dev argues coding models now beat nearly all engineers in every branch of software — zack_overflow · 2026-10-16
- Prediction: Chinese frontier models could match GPT-6 by March 2027 — teortaxesTex · 2026-10-16
- Liquid AI's d1-ultra, #1 on Artificial Analysis Decision Index, lands on Vercel — JosephJacks_ · 2026-10-16