PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud
Chenghua Wang, Daliang Xu, Dongqi Cai, Duojin Sun, Hao Zhang, Haoze Qian, Huaiyuan Zhang, Jinshuo Cui, Junbo Cui, Kezhao Zhao, Longxi Gao, Mengwei Xu, Rongjie Yi, Tam Sikyuen, Tianyue Zhang, Weikai Xie, Xuanzhe Liu, Yingying Qin, Yiwen Lu, Yuan Yao, Yuezhi Zu, Yunhan Guo, Yuxin Zheng, Ziqi Guo
cs.AI, cs.RO
2026-08-04
PhyAI, a unified runtime from BUPT and collaborators, runs VLA and world-action robot policies across onboard, edge, and cloud with up to 4.65x speedup over official implementations.
A single physical-AI policy has to run through very different scenarios during its lifetime: offline evaluation, cloud reinforcement-learning rollout, shared edge GPU serving, and onboard inference on the robot itself. All of these share the same checkpoint and action semantics, but teams typically end up maintaining a separate inference program for each one: an offline eval script, a cloud batch-serving stack, an edge deployment build. Each path gets optimized and validated on its own, and nothing learned in one setting automatically carries over to another. PhyAI, built by a team from Beijing University of Posts and Telecommunications with collaborators from Nanjing University, Peking University, Tsinghua University, and ModelBest, sets out to cover all of these scenarios with one runtime.
PhyAI's core move is separating architecture-specific logic from shared execution services. Each model architecture, whether a VLA model like π0, π0.5, or GR00T, or a world-action model like Cosmos3, keeps its own conditioning, solver state, caching, and output-conversion logic inside a model adapter. Graph execution, kernels, memory and cache management, and parallel scheduling live in the shared runtime instead of being reimplemented per model. The same codebase then runs both VLA and world-action models, on one or many GPUs, across onboard, edge, and cloud deployment. The team demonstrated how cheap adding a new architecture is by wiring in MiniCPM-Robot on the day it was released.
The optimization techniques underneath include kernel fusion (for example, fusing π0.5's Q/K/V projections into a single GEMM, and merging the MLP gate and up projections), CUDA Graph replay (capturing the repeated control flow and tensor state across π0.5's ten-step Euler loop once and replaying it), shape- and hardware-aware kernel selection, and offline quantization (W4A16, W8A16, W8A8, W4A8). The team also introduces a control-time Roofline, an analysis tool for deciding whether a given control loop is inference-bound or environment-bound. If the environment is already the slower half of the loop, shaving more time off inference stops paying off, and that slack can instead buy a bigger model or a cheaper accelerator.
Across π0, π0.5, GR00T N1.7, and MiniCPM-Robot, PhyAI beats the official implementations by 1.40x to 4.65x. The comparison isn't fully precision-matched, and specialized runtimes such as FlashRT and vla.cpp still win in several configurations; PhyAI's goal is a single runtime with competitive latency everywhere, not the fastest number in every case.
On Cosmos3-Nano-Policy-DROID, running on eight H20 GPUs with CFG=2 and TP=4, PhyAI cuts latency from 2.46s to 1.18s, a 2.08x speedup. Detailed profiling shows why different models need different execution policies: on a Hopper-class GPU at batch size one, π0.5's action expert accounts for just 8.8% of estimated FLOPs but 57.2% of measured latency, because its ten-step iterative loop launches many small kernels; at batch size 32, that share drops to 13.5% and throughput reaches roughly 100 samples/s. Cosmos3, by contrast, stays generation-dominated and gains only 14.3% throughput going from batch size 1 to 16. Applying the control-time Roofline to four LIBERO suites shows that π0.5's measured inference is already faster than the simulator's environment execution (environment-bound), while Cosmos3 remains inference-bound, meaning further speeding up π0.5 has limited payoff while Cosmos3 still has real headroom.
Embodied-AI teams commonly maintain four separate code paths for the same policy: eval scripts, cloud RL training, edge serving, and onboard deployment, each optimized and debugged independently. PhyAI shows these can share one runtime, with new architectures added purely through adapters. Wiring in MiniCPM-Robot on release day is a concrete demonstration that onboarding a new model doesn't have to be expensive. The control-time Roofline is also independently useful: it gives teams a quantitative answer to whether it's still worth optimizing inference speed, instead of guessing at the bottleneck. For anyone building embodied-AI infrastructure, this is a path toward not reinventing the wheel for every model and every deployment target.
The authors are upfront that PhyAI isn't the fastest option everywhere; runtimes built for one specific model and one specific chip, like FlashRT and vla.cpp, still win in several configurations, since PhyAI trades some peak performance for generality. The reported latency numbers cover GPU execution time and static batches only. They exclude observation capture, network transfer, request queueing, actuator handoff, tail latency, closed-loop RL rollout, and actual robot task success, and the paper explicitly says these numbers characterize runtime performance rather than end-to-end robot success or safety. The speedup comparisons also aren't fully precision-matched (some results use explicit FP8), and the authors admit in their future-work section that Thor still lacks architecture-specific kernels for many common shapes, which currently fall back to generic framework implementations.