SLA: Scaling Spatial-Language-Action Model via Continual Pre-training
Fuhao Li, Wenxuan Song, Qizhou Huang, Ziyang Zhou, Jingbo Wang, Junwu E, Pengwei Wang, Yan Wang, Shunbo Zhou, Haoang Li
cs.RO
2026-11-06
SLA continues pre-training from π0.5, aligning visual tokens to a frozen 3D teacher. Eight real-robot conditions average 83.8%, 24.0 points above π0.5.
Vision tokens in most vision-language-action models come from 2D image-text pre-training, and 3D supervision usually waits until post-training. Grasping and handover depend on the relative pose of the gripper and the object. Depth maps and point clouds are a weak foundation: measurements are incomplete, calibrations disagree, and many public trajectories have no depth. Spatial Forcing aligns visual tokens to an external 3D representation so deployment needs no extra 3D input, but that alignment runs only in post-training, so the spatial skill stays tied to the task.
SLA keeps training from Physical Intelligence's π0.5, moves the alignment into the base model, and leaves the same loss on during downstream adaptation.
Actions are conditioned on visual and language tokens. The action expert emits continuous chunks with flow matching. If intermediate visual tokens carry no 3D structure, the action head has nothing geometric to read. A probe freezes a VLA and trains only a DPT depth head: unaligned embeddings do not decode into depth, and aligned ones do.
Training adds a branch that is removed at deployment. Frozen OmniVGGT reads multi-view RGB. When camera intrinsics, extrinsics, or depth are present, its GeoAdapter writes them into the features; missing annotations become placeholder tokens. Features plus positional encodings are bilinearly resampled onto the VLA token grid. The PaliGemma backbone has 18 layers. Visual tokens from every layer share a LayerNorm and a two-layer MLP, then a per-token, per-channel sigmoid gate, and the gated features are summed. Gate parameters are shared across layers. Scenes and robot bodies differ enough that one hand-picked layer fits only part of the mix, so the gate sets layer weights from the current features. The alignment loss is one minus cosine similarity, scaled by a weight α and added to the action loss. The teacher stays frozen. At inference the teacher, projector, and gates are gone. The policy reads neither depth nor calibration, and proprioceptive state is supplied as usual.
Joint angles do not transfer across embodiments, so observations use absolute end-effector poses and targets use a translation plus a rotation vector relative to the current pose. Relative rotations are composed in SO(3), which avoids Euler wraparound near ±2π. Each training sample also drops the whole state-token span with some probability, so the policy cannot replay a demonstration from joint angles alone. Six public sources contain about 1.714 million episodes and 14,330 hours before filtering. The mix keeps two-finger grippers, drops mobile bases and downstream benchmark data, and subsamples repetitive simulation. Some InternData-A1 tasks keep 3% of episodes. RoboCasa MimicGen keeps 0.5%.
Continual pre-training runs for 36k steps at an effective batch of 4,608 and takes about 110 hours on 48 H100s. Post-training keeps the alignment loss. LIBERO-Plus stresses observation and language perturbations. RoboTwin trains on clean demonstrations and tests randomized scenes (Clean2Random). RoboCasa-Panda is kitchen manipulation with 50 demonstrations per task. The real-robot suite is eight private conditions.
| Benchmark | Comparison | SLA |
| LIBERO-Plus weighted SR | π0.5 84.4%, Spatial Forcing 85.1% | 87.4% |
| RoboTwin Clean2Random | π0.5 46.0%, ME-Dex-1.0 68.1% | 72.3%, avg 79.2% |
| RoboCasa-Panda, 50 demos/task | Cosmos 67.1%, Spatial Forcing 67.6% | 68.4% |
| RoboDojo-Real SR | π0.5 12.8%, InternW0-∆ 31.7% | 33.9%, score 42.3 |
| Eight real-robot conditions | π0.5 59.8%, Spatial Forcing 68.0% | 83.8% |
SLA is highest on every real condition: bimanual vegetable handover 94%, retrieving thrown fruit 82%, a target moved during the grasp 76%, a Franka after 100 Piper demos plus 10 Franka demos 78%, and black tape held out of downstream training 80%. Spatial Forcing, which aligns only in post-training, falls below π0.5 when table height changes and when the object was unseen (64% vs 68%, 56% vs 60%). On RoboCasa, 10 of 24 tasks exceed 80%, including 100% on TurnOffMicrowave.
With alignment in both stages, Clean2Random rises from 49.2% at 5% of the pre-training data to 72.3% at the full mix. With no alignment it moves only from 43.8% to 48.2%. At full data, alignment only in pre-training reaches 63.2%, and only in post-training 57.1%. The best fixed layer, layer 12, scores 67.3% Clean2Random. Adaptive fusion of all 18 layers scores 72.3%, above layer 18 alone at 64.6%. State dropout at 25% averages 79.2% across the two RoboTwin settings; at 0% the average is 38.1%. Rotation vectors average 79.2%, ahead of quaternions at 74.3%, Euler angles at 65.5%, and Rot6D at 62.9%. All four variants compose rotations in SO(3).
The π0.5 inference graph can stay unchanged. The 3D teacher, projector, and gates exist only in training. Across eight real-robot conditions the average is 24.0 points above π0.5. RoboCasa reaches 68.4% with 50 demonstrations per task, while other policies in the same table that use 300 to 3,000 demonstrations top out at 66.4%. Extra data without alignment barely moves randomized scenes. Alignment in both stages is what turns added trajectories into Clean2Random success.
LIBERO-Plus is only 3.0 points above π0.5, and its language split moves only from 80.8% to 83.5%. The RoboTwin average is 79.2% against 78.9% for ME-Dex-1.0, and the clean setting still trails, 86.0% to 89.6%. On the simulation boards, the gains sit mostly in scene randomization and observation shift.
The paper has no limitations section. The headline recipe binds spatial alignment, 25% state dropout, and rotation vectors together. On one continual-pre-training checkpoint, raising dropout from 0% to 25% jumps the RoboTwin average from 38.1% to 79.2%, and higher rates fall back. That gap is larger than most leaderboard margins. The scaling study toggles alignment but does not cross it with dropout, so the 24.0-point real-robot gain cannot all be assigned to the 3D teacher.
Against π0.5, noise rises from 89.0% to 95.4% and background from 94.1% to 98.0%, while the robot perturbation moves from 73.6% to 73.8%. Against Spatial Forcing, SLA is higher on camera and noise (82.2% vs 70.2%, 95.4% vs 86.0%) and lower on robot, language, and layout (73.8% vs 79.3%, 83.5% vs 85.6%, 86.2% vs 88.3%). The 0.3-point edge over robot-only Qwen-RobotManip is about 30 trials inside 10,030 rollouts. RoboCasa's higher-budget numbers come from other policies, not from a data curve of this one.
RoboDojo-Real adds a memory module and still feeds historical images at inference, so that run no longer uses the original compute graph. On the ARX X5, SLA scores 50.3 with 36.7% success, behind InternW0-∆ at 59.7 and 50.0%. Piper X success is 10.0%. The calibration ablation uses a separate, privately collected RoboTwin set, 46.0% with calibration versus 42.8% without, and those rates are not the main 79.2% result. The 1.714 million episodes are counted before filtering. The number actually trained is not reported. Coverage stops at two-finger grippers on fixed bases.