ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models
Manan Tayal, Akshay Nambi
cs.RO, cs.AI
2026-09-02
A model-free HJ critic gates PPO: task reward in the feasible set, recovery outside. Versus SafeVLA, safety cost falls 57% on average and success rises 0.13 on five sim benchmarks.
VLA fine-tuning usually chases task reward only. One collision or one trip into a hazard zone ends the job, in a kitchen or on a line. The stock fix is a Lagrangian on a Constrained MDP: keep expected cumulative safety cost under a budget via a dual penalty. SafeVLA is that recipe. A soft constraint on the expectation does not police a single trajectory, and a global penalty pushes the policy conservative, worse on long-horizon tasks.
Supervision is the other hole. Visual domains rarely have per-step safety labels, and free-form VLM scores are too noisy to train a critic on.
ShieldVLA treats safety as "can this state stay safe forever," not as a running deduction. Hamilton-Jacobi reachability defines a safety value \(V^s(s)\): from \(s\), under worst-case future constraint, can the margin stay non-negative. \(V^s \ge 0\) means some policy remains in the safe set; \(V^s < 0\) means every continuation eventually violates. Fisac et al. (2019) give a discounted safety Bellman operator that mixes with a min instead of an additive backup. It is a \(\gamma\)-contraction and can be learned model-free with TD.
Three pieces.
Safety critic. A vision encoder plus MLP, separate from the policy, maps observation and action to \(Q^s\). It shares a replay buffer with PPO and takes \(K\) off-policy TD steps per PPO iteration, targeting the fixed point of the discounted safety operator. Terminals do not bootstrap. \(Q^s > \delta\) is feasible; \(\delta\) is a noise buffer.
Gated update. In the feasible region the update is ordinary clipped PPO on task reward, identical to unconstrained fine-tuning. In the infeasible region the task term is dropped and a deterministic policy gradient raises \(Q^s\), pushing the mean action onto the safe manifold. No separate recovery policy. The dual \(\betat\) multiplies only infeasible transitions and climbs against the cost budget. The gate \(\zeta\) is non-differentiable and used as a stop-gradient sample mask.
VLM rubric scores. Frozen Qwen3-VL-8B rates each frame on several rubrics in \([0,1]\); a weighted sum is the raw severity. A binary episode-level safety bit calibrates that severity through Platt scaling (two-scalar logistic) into a signed margin \(\hat\ell\). Default five axes: proximity, heading, contact, occlusion, clearance. Static rubrics miss rare failures; Refinement-through-Differentiation has Qwen2.5-VL-72B inspect rank-inverted frame pairs and propose new criteria. The VLM runs only at train time. Deployment is a single VLA with no runtime shield.
Five environments, 200 evaluation episodes. Baselines: unconstrained VLA, SafeVLA (PPO-Lagrangian), VisionCBF. Dubins-VL also has an HJ oracle with ground-truth signed distance. Success requires reaching the goal with zero violations.
| Method | Dubins SR/CSC | TurtleBot | CHORES Nav | CHORES Fetch | Franka |
| Unconstrained | 0.08 / 45.7 | 0.30 / 22.2 | 0.45 / 13.2 | 0.15 / 14.1 | 0.10 / 45.0 |
| SafeVLA | 0.22 / 2.66 | 0.34 / 14.6 | 0.59 / 1.82 | 0.34 / 8.98 | 0.34 / 13.4 |
| VisionCBF | 0.18 / 1.31 | 0.38 / 12.1 | 0.24 / 1.22 | 0.18 / 6.34 | 0.29 / 6.8 |
| ShieldVLA | 0.32 / 1.3 | 0.54 / 6.90 | 0.68 / 1.44 | 0.49 / 5.50 | 0.45 / 3.50 |
The paper reports a 57% average drop in cumulative safety cost versus SafeVLA and +0.13 success. Three-seed standard deviations are small relative to those gaps. On Dubins the HJ oracle hits SR 0.38 and CSC 1.13; ShieldVLA does not reach that task ceiling. One sentence in the main text says "zero cost," which disagrees with the table's 1.3; trust the table.
Ablations. Same HJ critic and gate, swap the continuous VLM margin for a binary collision indicator (+1 / -5): TurtleBot success falls from 0.54 to 0.31, Franka from 0.45 to 0.25. With no gradient away from the boundary the gate closes too early. Keep the critic and replace the gate with a Lagrangian penalty at every state: TurtleBot SR 0.33, CSC 7.80, against gated 0.54 / 6.90. The Pareto gain is the gate, not the critic alone.
At test time, color and lighting change and weights stay frozen. On TurtleBot, SafeVLA's CSC rises from 14.6 to about 22.7; ShieldVLA from 6.9 to 10.7 on the worst (+All) split. On Safety-CHORES Nav, ShieldVLA's CSC falls. A 2B scorer mode-collapses on tail frames; 8B is what lets RTD criteria fire. Tail AUROC on stratified TurtleBot poses goes from 0.500 to 0.608.
For anyone already fine-tuning OpenVLA or SPOC-style backbones, this swaps a global \(\lambda\) for an intervention that fires only when the critic calls the state infeasible. No shield and no VLM call at deploy time. Supervision is an episode-level incident bit plus a written rubric.
The bill is real. Safety-CHORES Fetch is about 700 GPU-hours for one environment; scoring and RTD are cheaper (about 26 minutes per environment at 8B). Everything is simulation. This is reachability from control, wired into VLA PPO. It is not a formal safety certificate.
The authors name two: scorers under 8B mode-collapse on tail frames; if unsafe events are very rare, near-boundary exploration is too thin for a stable critic. They point next at hardware and a larger scorer.
No physical robot. Safety numbers are empirical; learned \(Q^s\) carries finite-data and VLM noise. The gated estimator is biased near the boundary. \(\delta\) and \(c{\max}\) are hand-set per environment. VisionCBF has lower CSC than ShieldVLA on CHORES Nav (1.22 vs 1.44) but success 0.24 against 0.68; Pareto comparison needs both axes. Dubins-VL is not a full VLA, it is ResNet-18 plus frozen CLIP as a controlled benchmark. OOD only retints the renderer; geometry, dynamics, and cameras stay put.