RT-SAFE benchmark: real-time embodied agents crash 12.3× more, safe success drops to 0.7%
Lianhuiq · x · 2026-10-13
Researchers from Cornell and UCSD introduced RT-SAFE, a simulated urban benchmark that evaluates embodied agent safety while the world keeps moving during inference—navigation tasks with moving pedestrians, hazards, and traffic rules.
Key findings:
- Static vs real-time evaluation: task success looks fine (91.3% → 94.1%), but safe completion collapses from 19.8% to 0.7%
- Real-time conditions cause 12.3× more collisions per episode than paused-world tests
- Counterintuitively, increasing reasoning effort barely moves task success (94.1% → 94.8%) while collisions surge from 40.7 to 62.0 per episode—decision latency itself becomes a source of physical risk
Takeaway: standard task success masks serious safety failures; embodied safety depends on both decision quality and decision latency.
More from Embodied
- Amazon now has over 1 million robot workers. What's next for human employees? — lulzxdxdxd · 2026-10-13
- Chris Paxton on IROS: dexterous, contact-rich manipulation is now robotics' frontier — chris_j_paxton · 2026-10-13
- Scoble hails non-invasive BCI from Phantom Neuro after quadriplegic user demo — Scobleizer · 2026-10-13
- Most iPhone Duo apps look awful for now: old-SDK apps sit centered with empty space until iOS 27 SDK — steipete · 2026-10-13
- Dev to test $13 voice board with Codex and GPT Live in DIY AI device — OpenAIDevs · 2026-10-13
- RT-SAFE: Agents Hit 94.1% Task Success but Only 0.7% Safe Success — Lianhuiq · 2026-10-13