4,700-vote marketing eval: GPT-6.2 Sol beats Opus 5.5 at 58% win rate and a quarter of the cost
boringmarketer · x · 2026-11-21
boringMarketer ran a deep marketing eval with Claude Opus 5.5, GPT-6.2 Sol and Gemini 4 Neon doing real marketing for real brands: 4,700+ head-to-head votes, judges from 5 labs, none scoring its own lab's model.
- Surprise winner: GPT-6.2 Sol took 58% of votes across 15 marketing tasks at $0.04 per piece — less than half of Opus's cost and a fifth of Gemini's. Opus scored 52% ($0.10), Gemini 40% ($0.21).
- Task breakdown: Sol dominates campaign plans (91%), cold emails (92%), long sales pages (78%), positioning briefs (75%), and makes zero unsupported brand claims. Opus wins video editing (77% — Sol's edits look like slideshows) and rewriting a flopped post.
- Key lesson: no single model wins everything; pick per task.
More from Models
- Kimi K3.1 Pro jumps from #37 to #9 on Arena Alignment Index, deceptive completion down to 8.6% — arena · 2026-11-21
- Moonshot ties Anthropic for #2 lab on Arena Alignment Index, tops Chinese labs — arena · 2026-11-21
- Arena Alignment Index update: OpenAI sweeps top seven, Claude Fable 5.2 debuts at #8 — arena · 2026-11-21
- Asking Astra 6.2 to Review Code Written by Opus 5.5 Yields Almost Nothing — rickasaurus · 2026-11-21
- Parallel Search tops Artificial Analysis Search Index at 83, with fast mode at $0.014 per task — ArtificialAnlys · 2026-11-21
- Google Black Friday: up to 50% off AI Plus and annual Pro plans through Nov 26 — gaganghotra_ · 2026-11-21