SAE weight editing steers robot behavior at inference time, IIIT Hyderabad paper lands IJCAI Safe Physical AI workshop
ponguru · x · 2026-10-18
A paper from IIIT Hyderabad, Controlling Robot Behavior at Inference Time: Weights based SAE steering in Vision Language Action models, was accepted at the IJCAI Safe Physical AI Workshop. The project began in the authors' Spring 2026 Responsible & Safe AI course.
The method works in three stages: training a Matryoshka SAE on the FFN of layer 31 of a VLA model to get a sparse latent space; projecting FFN vectors into vocabulary space via the LM head to select SAE features tied to concepts like "task", "slow", "push" and "destroy"; and applying signed feature amplification at inference time, reconstructing the steered FFN weights as a temporary weight edit.
In simulated manipulation experiments, steering reduced execution time and improved task success rate, offering a way to control robot behavior safely without retraining. A 5-minute podcast discussion of the paper is also available.
More from Embodied
- MIT's SANDO Drone Planner Avoids Every Moving Obstacle Across 12 Real Flights — Olivier__OG · 2026-10-18
- SimInfinity: open-source robotic simulation benchmark with 10K real-world tasks — joemeno · 2026-10-18
- Robi: a German/English voice companion demo built on Gemma, Tyger and Humaneness Voice TTS — realmrfakename · 2026-10-18
- Munich's Plim unveils wheeled home-care humanoid controllable from 9,000 km away — CyberRobooo · 2026-10-18
- Musk says Optimus and Cybercab will make up the vast majority of Tesla's value — XFreeze · 2026-10-18
- NTU-led survey reviews nearly 200 papers on affordance learning for embodied AI — jiqizhixin · 2026-10-18