Benchmarking 18 LLMs for 'AI Slop': Over-Optimization Increases Clichés
penguinothepenguin · reddit · 2026-08-02
A developer benchmarked 18 major AI models to quantify and compare how much 'AI slop' (clichéd, templated writing) they produce.
Methodology:
- Baseline: Established a human baseline using corpora across emails, social media, chats, and essays.
- Tasks: Hand-wrote 112 writing scenarios, generating outputs from all models at default settings.
- Axes: Scored across 5 dimensions: Conciseness, Templating, Rhythm, Tells (overused vocab/constructions), and Human Preference. Notably, no LLMs were used as judges.
Key Findings:
- Human preference heavily influenced the final rankings. For instance, Fable ranked #2 on mechanical metrics but dropped to last place overall due to low human preference scores.
- The author suggests that recent models have become overly benchmark-optimized, paradoxically leading them to produce more AI slop rather than less. This highlights the continued importance of good prompting and orchestration.
All methodologies, outputs, and code are open-source.
More from Models
- GYRO quant of Qwen3.8-Flash-Next tested: near Q3 quality at much smaller size — PlasticRevenue4601 · 2026-10-16
- Open source is so back: dozens of open models dropped within 48 hours — beffjezos · 2026-10-16
- Gemini users hit usage caps on 3.1 Pro and 4 Argon, resets not until Oct 16 — R3d-Gr33n-Blu3 · 2026-10-16
- Gemini desktop app can't accept images — an image model that can't take images — Rebl · 2026-10-16
- Apollo Research: GLM-5.3 unprompted reasons about its actions becoming training data — _lewtun · 2026-10-16
- Opus 5.5 turns heads with surprisingly polished vertical video demo — FinanceYF5 · 2026-10-16