Artificial Analysis and HeyGen Launch Code2Video-AA Marketing Video Generation Benchmark

Artificial Analysis has partnered with HeyGen to release the Code2Video-AA leaderboard—an independent implementation of the HeyGen Code2Video Bench—evaluating models' ability to generate product/brand marketing videos with code. It covers 27 models and 114 design briefs, with the full leaderboard and methodology publicly available.

Confirmed

• Evaluation pipeline: each model reads a design brief and writes a HyperFrames HTML composition in a single shot (no tools), rendered offline into a video; a render failure means losing all comparisons. Judge Gemini 4 Argon views the same brief at 8 frames per second, producing 35,000 blind judgment calls to compute Elo.
• Overall leaderboard: Claude Opus 5.5 leads with a 93-point Elo advantage, but at $3.08 per task—roughly 10x some models on the same board.
• Axis breakdown: Opus 5.5 tops all five judging axes on Elo, leading runner-up Claude Sonnet 5.5 (max with fallback) by 128 points on the craft axis; the temporal consistency axis shows the least discrimination.
• Cost-effectiveness: the Elo-vs-cost Pareto plot (8 models featured) shows high rank doesn't require high cost—GPT-6.1 Sol (max) ranks fourth (1369) at just $0.31 per task, about one-tenth of leading Opus 5.5.
• Sample gap: on the same design brief, submission quality varies drastically across tiers: Claude Opus 5.5 (1536), DeepSeek V4.1 Flash (1053), MiniMax-M3 (724), and the lower-scoring Nemotron series.

Why it matters

This is the first leaderboard to quantify the emerging capability of "generating marketing videos with code" via standardized blind evaluation, while factoring cost into the comparison—showing that premium flagship models aren't the only option. For model vendors, the per-axis data also pinpoints specific weaknesses such as craft quality and temporal consistency. AA credited HeyGen (original authors of Code2Video Bench) in the release.

2026-10-27 ~ 2026-10-27 · 6 related posts

• [source] Code2Video-AA: Claude Opus 5.5 Tops Marketing Video Coding Bench by 93 Elo at 10x the Cost — ArtificialAnlys · 2026-10-27
• How Code2Video-AA Works: Single-Shot Generation, Blind Pairwise Elo Judging — ArtificialAnlys · 2026-10-27
• Opus 5.5 Sweeps All Five Judge Axes, With a 128-Point Craft Lead — ArtificialAnlys · 2026-10-27
• [source] Code2Video-AA Cost Curve: 8 Models on the Pareto Frontier, High Elo Needn't Be Pricey — ArtificialAnlys · 2026-10-27
• Code2Video-AA Samples: Same Brief, Four Models, Massive Quality Gaps — ArtificialAnlys · 2026-10-27
• [source] Code2Video-AA Full Leaderboard and Methodology Now Live — ArtificialAnlys · 2026-10-27