Realtime-Bench: best frontier models reliably complete only 36% of real-time tasks
ChengleiSi · x · 2026-10-24
jiaxinpei's team released Realtime-Bench, a carefully curated benchmark that systematically tests both native realtime models and general VLMs on their ability to think and act in real time — motivated by tasks from playing games to grabbing concert tickets that require real-time interactive decisions. Results are sobering: the best model reliably completes only 36% of tasks, exposing a large gap between real-time interactive capability and offline model performance.
More from Research
- Stanford-led paper traces 'bigger is better' to data-induced competition for neurons — niloofar_mire · 2026-10-24
- Frequency drives conventionalization: higher-exposure communities conventionalize more words — abenitezburraco · 2026-10-24
- GeoWind2Plan: seconds-scale 3D urban wind prediction for energy-efficient drones — AnimaAnandkumar · 2026-10-24
- Flowing water and lava come to 3D Gaussian Splatting scenes — jonstephens85 · 2026-10-24
- Immorta Bio's AI-designed 9-amino-acid peptide SenoMune-0257 targets senescent cells — DavidmComfort · 2026-10-24
- France rebuilt its tax code as MLang, a formally verified open-source compiler — fkasummer · 2026-10-24