Inferring Physical Properties via Motion Probes

新智元 · wechat · 2026-07-16

PhyMAGIC aims to solve the issue of "a single image being insufficient to understand physical properties." Instead of directly guessing object parameters, it generates targeted motion probe videos, then uses a Vision-Language Model (VLM) to extract physical evidence from these motions and iteratively refine its judgments.

Methodology Highlights

Uses an image-to-video model to generate motion probes like free fall, collision, squeezing, rotation, and side-pushing.
Extracts key frames from the video for the VLM to infer parameters such as material, mass, density, Young's modulus, Poisson's ratio, and yield stress, estimating confidence levels for each.
Automatically rewrites motion commands for low-confidence parameters to generate more targeted evidence, forming an "observe-infer-re-observe" closed loop.
Combines physical parameters and simulator execution parameters into HPP, integrating 3D Gaussian Splatting with MPM simulations to render dynamic 3D objects viewable from multiple angles.

Results and Limitations

Tested on PhysGaussian, PhysGen, and internet single-image scenarios, PhyMAGIC outperformed multiple baseline methods in CLIP similarity and Image-Motion-FID.
61 participants also favored it in ratings for physical plausibility and text consistency.
However, it remains limited by single-image 3D reconstruction quality, VLM physical parameter estimation accuracy, and insufficient support for complex joint structures and multi-object interactions.

Original post →

Inferring Physical Properties via Motion Probes

More from Multimodal