RT-SAFE Benchmark Exposes Real-Time Safety Blind Spot in Embodied AI

Cornell and UC researchers (Tianruo Rose Xu, Jiawei Ren, et al., advised by Lianhui Qin) released the RT-SAFE benchmark on October 13 (arXiv:2610.09294), highlighting a core challenge for embodied AI: reasoning itself consumes safety-critical time—the physical world keeps changing while the agent thinks, making real-time safety a key blind spot. The paper, website, and code are all public.

Confirmed

RT-SAFE is a real-time safety evaluation benchmark for physical-world AI agents, building urban scenarios with real-time constraints in a simulated city that combine navigation tasks, moving pedestrians, environmental hazards, and traffic rules to assess embodied agents' safety under "the world doesn't stop" conditions
Key numbers: 8 VLM agents achieved task completion rates as high as 94.1%, but safety success rates as low as 0.7% in the hardest setting; collision rates in real-time environments reached 12.3x those of static tests; task success rates under static vs. real-time evaluation compared at 91.3% vs. lower
Results show agents can complete tasks while massively violating safety constraints along the way; the longer an agent thinks, the higher the collision risk

Why It Matters

Traditional static evaluations fail to capture real-world time pressure; RT-SAFE fills the gap in real-time safety evaluation
The huge gap between task success and safety success (94.1% vs. 0.7%) reveals a systemic deficiency in current embodied agents on the real-time safety dimension, with cautionary implications for embodied AI deployment and evaluation paradigms

2026-10-13 ~ 2026-10-13 · 5 related posts

Primary sources

RT-SAFE benchmark: real-time embodied agents crash 12.3× more, safe success drops to 0.7% — Lianhuiq · 2026-10-13 03:13 (UTC+08:00)
RT-SAFE benchmark: agents hit 94.1% task success but only 0.7% finish safely in real time — Lianhuiq · 2026-10-13 03:17 (UTC+08:00)
RT-SAFE: Agents Hit 94.1% Task Success but Only 0.7% Safe Success — Lianhuiq · 2026-10-13 03:11 (UTC+08:00)

[source] RT-SAFE: Agents Hit 94.1% Task Success but Only 0.7% Safe Success — Lianhuiq · 2026-10-13
[source] RT-SAFE benchmark: real-time embodied agents crash 12.3× more, safe success drops to 0.7% — Lianhuiq · 2026-10-13
[source] RT-SAFE benchmark: agents hit 94.1% task success but only 0.7% finish safely in real time — Lianhuiq · 2026-10-13
RT-SAFE targets real-time safety blind spot in embodied AI — repligate · 2026-10-13

1 near-duplicate retellings: Lianhuiq