DialogueBench: Gemini Neon edges GPT-6.2 Sol 67.9% vs 64.5% at video dialogue editing
MatthewBerman · x · 2026-11-20
Matthew Berman published the Scarlet Dialogue Eval, a new benchmark testing models' ability to accurately edit dialogue in video, pitting Gemini 4 Neon against Codex GPT-6.2 Sol on a 20-minute talking-head edit (average of 2 runs).
- Overall: Gemini Neon 67.9% vs 64.5% (+3.4 pts)
- Hard mistakes /10 min: 13.3 vs 14.8 (10% fewer)
- Edit time: 7.7m vs 11.7m (34% faster)
- Gemini notably better at No wrong cuts (67.2% vs 34.7%); both weak at No mid-word cuts (24.6% vs 42.1%, lower is better); Editorial Match nearly tied
Takeaway: Gemini Neon wins overall and is much faster, but mid-word cuts remain unreliable for both.
More from Models
- HardGeoBench: top AI models now pass@1 above 50% locating metadata-free photos — Afinetheorem · 2026-11-20
- GPT 6.2 Astra tops Kevin Bryan's HardGeoBench as best OpenAI model yet, with caveats — Afinetheorem · 2026-11-20
- GPT-6.2 Sol hits 8.7 min per task, user bets ultrafast tier reaches $100 plan soon — haider1 · 2026-11-20
- Gemini freezes the browser for minutes on long math derivations — jasondeanlee · 2026-11-20
- Dev known for tiny-LLM work recounts dropping out, claims near-SOTA models under 8GB RAM — gnukeith · 2026-11-20
- Dev calls GPT 6.2 Sol 'as dumb as Luna, maybe dumber' for coding — burkov · 2026-11-20