4,700 blind judgments show DeepSeek/GLM with a custom design harness match Claude Code + Sonnet 5.5
JannVanDam · reddit · 2026-10-30
The author built Rivermark, an agentic harness for social/marketing/print design, and benchmarked 112 small-business design requests using 4,700+ pairwise blind human judgments (Bradley-Terry rankings).
Key findings
- Open models with Rivermark — DeepSeek v4.1 Flash (0.56) and GLM 5.3 Flash (0.57) — match or beat plain Claude Code + Sonnet 5.5 (0.51)
- Rivermark + Sonnet 5.5 jumps to a dominant 0.73, a surprisingly large lift even for a frontier model
- Control harness 'pi' scores only 0.47 (DeepSeek) and 0.30 (GLM FlashX), showing the gains come from the harness
Caveats
- The harness embeds design domain knowledge, is tuned for the rivermark.ai product's fast-iteration workflow (using the pricier GLM FlashX endpoint), and isn't a general-purpose agent
- An open-source release is planned
Takeaway: domain-specific harnesses let cheap models approach flagship quality at a fraction of the cost; a blog post and further validation are underway.
More from coding & agent
- iOS agent factory autonomously fixes its own stale bundle ID bug via self-improvement PR — vikvang1 · 2026-10-30
- Global Law as one Git history: HN #1 repo makes world law queryable by AI agents — SnooPeripherals5313 · 2026-10-30
- Fine-tuning Liquid d1 decision models on screenshots: 51% to 96% accuracy in ~20 minutes — helloiamleonie · 2026-10-30
- Measuring compaction on a 400k-token session: 38s and $1.99 per run — brandon_galang · 2026-10-30
- pstack's /statechart skill ships with a mini TLC compiler that helped find real distributed-systems bugs — DavidKPiano · 2026-10-30
- First video demo: using Grok bot for daily workflow, outer vs inner loop — fbaptista · 2026-10-30