A code-based AI-slop score and 2,621 blind human votes disagree across 70 models
penguinothepenguin · reddit · 2026-11-20
The Slop Index benchmark scores 70 models across 77,943 outputs (emails, Slack messages, social posts, short essays) two ways: a reproducible pure-code mechanical score (padding, templating, sentence rhythm, known AI tells, measured against pre-2022 human writing) plus a 40%-weighted crowd Elo from a blind arena with 2,621 votes.
The disagreements are the interesting part (models with 190+ votes):
- Gemini 3.5 Flash: mechanical #32, combined #2 — readers keep picking it as least AI-like
- Qwen3.7 Max: mechanical #25, combined #65
- GPT-5.6 Sol: mechanical #15, combined #69
- Haiku 4.5: mechanical #47, combined #14
- Where they agree: Mistral Large is mechanical #1 and combined #5
The author's read: people react to something the mechanical axes miss, likely repeated constructions and a "helpful assistant" stance. Limitations: the code misses repeated constructions and everything is short-form, no fiction. Leaderboard, methodology, open dataset, and code are public; the project started as internal model selection for the Slashy email client.
More from Models
- Gemini 4 Neon lands #4 on ValsAI's RSI Index, 3.8 pts above Argon, as Google overtakes OpenAI — teortaxesTex · 2026-11-20
- GPT 6.2 credited with proving an open math conjecture despite poor communication — apples_jimmy · 2026-11-20
- Simon Willison visualizes Gemini 4 effort levels with the pelican-on-a-bicycle test — hugobowne · 2026-11-20
- OpenAI policy chief touts research transparency, but METR task-horizon gap is widening — soumitrashukla9 · 2026-11-20
- One year later, model naming is still a mess: the gpt-5.1-codex-max anniversary — heyneighbor · 2026-11-20
- Developer slams Grok models as 'garbage', shares coding agent's repetitive trial-and-error log — feregri_no · 2026-11-20