GPT-6.2 Astra takes #1 on LLM creative writing benchmark at 4.30; Qwen 4 Max leaps to third
sandersted · x · 2026-11-18
LechMazur updated his LLM Creative Story-Writing Benchmark (400 constrained story briefs, blind pairwise comparison with Thurstone scoring):
- GPT-6.2 Astra (high) takes #1 at 4.30, sweeping all 50 matched prompts against predecessor GPT-6 Astra; qualitatively it needs less explanation in endings, lets multiple characters shape outcomes, and leaves more room for mixed motives, surprise and humor
- Claude Fable 5.2 is second at 3.56
- Qwen 4 Max jumps 3.72 points (-0.17 → 3.55) into third; note 397/400 stories completed with 46 refusal retries
- ByteDance Seed2.1 Pro improves from -2.18 to -0.67
- Bottom of the table: GPT-OSS-120B (-3.29), Mistral Medium 3.5, Gemma 4 31B Reasoning
The benchmark (github.com/lechmazur/writing) requires incorporating 10 mandatory story elements, judged by multiple evaluator models reading matched stories in both orders.
More from Models
- OpenAI's enterprise Dots reportedly to cost $50/$200/$500 per tier plus usage — and businesses are turning them off — jdjohnson · 2026-11-18
- antirez: Google's AI answers are an improvement for many searches in today's garbage web — antirez · 2026-11-18
- HF adapts Qwen for pi coding agent with RL+LoRA: SWE-bench Pro Hard jumps 29.4%→62.7% — art_zucker · 2026-11-18
- Why AI Is Sycophantic: A Fully Honest AI Would Be Bad for Business — JFPuget · 2026-11-18
- Dev predicts Opus 5.5 will be remembered alongside Sonnet 3.5 and GPT-4 — inductionheads · 2026-11-18
- Ming-3.0-audio-flash Tops Audio Reasoning Benchmark at 99.8% Accuracy — alifcoder · 2026-11-18