Realtime-Bench Shows Best Frontier Model Reliably Solves Only 36% of Real-Time Tasks
ManlingLi_ · x · 2026-11-04
A new benchmark, Realtime-Bench, tests whether frontier models can reliably solve real-time interactive problems under end-to-end latency constraints.
- The best model, GPT-6-Astra (high), reliably solves only 36% of tasks
- Jev achieves sub-1s latency but passes just 8%
- The chart reveals a sharp accuracy/latency trade-off: GPT-6-Astra (low) 30%, Gemini 3.8 Flash (low) 22%, while Claude, Kimi K3, DeepSeek and others sit below 10% with 4-8s latencies
Takeaway: current frontier models remain far from reliable in genuine real-time interactive settings, and speed and accuracy remain hard to get together.
More from Models
- Leaked BenchCAD data shows OpenAI's internal model costs up to $37 per task, fueling $500 plan speculation — imjustnewatai · 2026-11-04
- 1,300+ graded word-association clues: Google models beat OpenAI's, both beat Anthropic's — Hardicecream17 · 2026-11-04
- Ex-OpenAI researcher Richard Ngo: alignment is the bottleneck to eliciting model capabilities — arthurcolle · 2026-11-04
- BallotBench sim trading: Gemini goes degen, tops leaderboard at $67 — mariofilhoml · 2026-11-04
- Claude Opus 5.5 hits 31.7T weekly tokens and $13M weekly spend, biggest launch yet — scaling01 · 2026-11-04
- Liquid AI launches d1-ultra, tied with SubQ Snap 1.1 and Elephant 9B at 65 on AA index — JosephJacks_ · 2026-11-04