πR^2: Enabling 40ms Real-Time Reactivity for Robot Manipulation Policies
CMU-SCS · hf · 2026-07-31
Current large model-based robot manipulation policies typically use open-loop action chunking. Due to high latency in the perception-to-action pipeline, they cannot react quickly to new sensory input mid-execution.
Researchers from CMU introduced πR^2 to achieve real-time reactivity while retaining large backbones. Core innovations include:
- Dual-channel Conditioning: Splits input into a fast channel (proprioception, updated every tick) and a slow channel (vision-language features, asynchronously updated), allowing reactions even with stale vision.
- Latency-adaptive Flow Schedule: Treats in-flight actions as inpainting conditioning, emitting actions in a single denoising step to adapt to varying hardware latencies.
Applied to GR00T-N1.7 on a real xArm6 platform, πR^2 achieves a 25Hz (every 40ms) replanning rate, 4x faster than the base policy. It improved success rates by up to 30% in real-world manipulation tasks.
More from Embodied
- Google Launches Gemini Robotics 2: Single Model Enables Multi-Robot Collaboration — DynamicWebPaige · 2026-07-31
- Google Launches Gemini Robotics ER 2 Embodied Reasoning Model — rseroter · 2026-07-31
- DIY Backpack Stereoscopic Camera Detects Mosquitoes on Orange Pi — yacineMTB · 2026-07-31
- Google DeepMind's Gemini Robotics 2 Enables Whole-Body Humanoid Control — The Verge AI · 2026-07-31
- Consumer Humanoid Robots Running Local LLMs at $5K Within 3 Years, Reddit Predicts — Terminator857 · 2026-07-31
- World's Largest Physical AI Data Engine Cranks Up Capacity — pduan · 2026-07-31