Towards AI builds writing benchmark: 4 LLM judges, 1000 human A/B tests on AI slop
AI Engineer · youtube · 2026-10-22
Louis-François Bouchard (Towards AI) breaks down why AI writing feels like slop: overused words, vague generalities, saying nothing, repetition, hallucination, and a voice that sounds like everyone at once.
The benchmark
- Tests whether models can write video scripts in his own voice: 10 scripts × 5 drafts per model × 4 LLM judges × 9 metrics, plus 1,000 human A/B checks
- Key finding: LLM judges favor outputs from their own model family (evaluation bias)
- Cost breakdown included
Practical tips
- Very long prompts full of examples
- One review pass from a strong model
- Many cheap drafts merged together
- Always a human edit at the end
More from coding & agent
- OpenLore gives every coding agent the same memory with one folder of Markdown — Arindam_1729 · 2026-10-22
- Walking bento box built with Claude Code + Blender Python, no MCP needed — sidahuj · 2026-10-22
- Claude Code Autonomously Generated 2,000+ Krea2 Images Overnight — reynadsaltynuts · 2026-10-22
- Indie dev building game #4 with AI starts with a central game asset management hub — ezshine · 2026-10-22
- Diffy: a cross-platform desktop code review app with a custom semantic diff engine — seatedro · 2026-10-22
- Open-Source Garmin MCP Connector Plugs Workout Data Into AI Agents — Vast-Grapefruits · 2026-10-22