Adaptive soft thinking lets each CoT step pick its own temperature, reaching 49.7 on OmniSpatial

Soft Spatial Reasoning

Rafi Ibn Sultan, Md. Sajid Alam Chowdhury, Saleh Zare Zade, Chengyin Li, Prashant Khanduri, Marco Brocanelli, Dongxiao Zhu

cs.CV, cs.AI

2026-09-30

A per-step softness controller over soft CoT states hits 49.68 on OmniSpatial, 2.75 over fixed soft thinking, leading open models on two zero-shot spatial benchmarks.

What problem this solves

Ask an LVLM where the red ball sits relative to the yellow chair, with two red objects in the scene, one on the chair and one in midair. Chain-of-thought forces the model to pick a single token at every step, so an early line like 'the ball on the chair' poisons every step that follows. The paper names this premature discretization. Soft thinking, which feeds a weighted mixture of token embeddings instead of one token, is the existing remedy, but a fixed mixing temperature runs into a dilemma: candidates sometimes extend one plausible interpretation and are worth keeping alive, and sometimes encode conflicting spatial relations that contaminate later reasoning. The right degree of softness changes from step to step.

Method

The framework post-trains Qwen3-VL-8B-Thinking with three parts:

• Soft CoT. Each reasoning step takes the top-5 candidate tokens, adds Gumbel noise to their log-probabilities (a differentiable sampling trick that lets RL backprop through sampling), and turns the perturbed scores into mixture weights through a temperature-controlled softmax. The weighted sum of embeddings becomes the next input state. The highest-weight token at each step also forms a discrete spine; when it emits </think>, generation switches to discrete tokens for the final answer.
• AdaptSoft. A two-layer MLP that reads a final-layer hidden state compressed to 8 dimensions plus the normalized entropy of the next-token distribution, and outputs a temperature of 0.5 + 0.4·tanh(u), bounded to (0.1, 0.9). Low temperature approximates picking a single token; high temperature spreads influence across candidates.
• Gradient-alignment training. The policy is trained with GRPO on answer and format rewards (soft-step likelihood ratios follow the Gumbel-reparameterized construction of Soft-GRPO). GRPO advantages are rollout-level, so they cannot say which step should be soft. AdaptSoft gets its own signal: 25% of each batch forms a reference subset whose output-layer gradient serves as a reference direction, and every soft step in the rest contributes its own gradient; the inner product with the reference becomes a step-level alignment score, and the loss is its negative. A step's temperature is rewarded when it points that step's update in the same direction as the batch. No intermediate reasoning annotations are needed.

Results

Matched-backbone comparison, weighted average accuracy:

On OmniSpatial, fixed soft thinking adds 1.01 points over hard thinking and adaptive softness adds another 2.75. Geometric Reasoning is the sharpest case: hard and fixed-soft both stall at 25.00 while adaptation reaches 34.84; Motion Analysis rises from 55.85 to 62.14. Against external models, 49.68 leads InternVL3-14B (45.94) by 3.74, SoFar (45.14) by 4.54, and LaCoT (45.66) by 4.02, the best non-proprietary result, with Traffic Analysis up 21.61 over the base backbone. Both zero-shot benchmarks hold the same rank among open models, 2.93 over LVR on SpatiaLab, and 48.71 also edges every proprietary reference in that table.

Ablations on Perspective Taking: removing gradient alignment costs 4.07 points (6.52 on Hypothetical reasoning), removing the hidden-state input 3.53, removing entropy 2.11, removing gradient centering 1.57. Step-level credit is doing real work.

The learned behavior has a counterintuitive part: higher predictive entropy triggers lower temperature (the median falls from 0.590 to 0.350, correlation -0.613). When candidates conflict, the controller narrows the mixture, which the authors read as limiting interference between conflicting interpretations; within a single entropy bin the 10th-90th percentile spread is still 0.18-0.20, so the hidden state carries information beyond entropy. Over training the mean temperature stays pinned near 0.5 while the min-max span grows from 0.23 to 0.58, and the within-trace spread (median 0.235) is about 2.5 times the spread between traces (0.093). What is learned is per-step differentiation, not a global shift.

Why it matters

This is the first soft-thinking framework aimed at LVLM spatial reasoning, but the reusable parts are more general. One is the problem statement: fixed softness is not enough, and a cheap control signal (hidden state plus entropy) suffices to adapt it. The other is the gradient-alignment objective, a template for training any step-level controller when only final rewards exist. Compute is modest: 8 H100s, 200 steps, about two epochs on 6,902 OmniSpatial samples, with code released. A 2.75-point matched-backbone gain is incremental, but the ablations attribute it cleanly.

Limitations

The authors state one themselves: AdaptSoft has no explicit measure of visual-evidence uncertainty, so softness reflects only language-side ambiguity.

Other open questions: a single 8B backbone and a single training set leave scaling behavior untested. On OmniSpatial it still trails the strongest proprietary models (o4-mini at 52.77), and on MindCube 38.13 sits nearly 9 points behind Gemini-2.5-Pro (47.05) and 6.6 behind Claude-4-Sonnet (44.75). The paper reports no inference cost (soft steps compute top-5 mixtures with Gumbel sampling) and no check on whether non-spatial capabilities degrade after post-training. The explanation for entropy-driven cooling is a plausible reading of correlational evidence rather than a verified mechanism.

Terms

• Soft thinking: intermediate reasoning steps use weighted mixtures of token embeddings instead of a single token; the final answer stays discrete.
• Premature discretization: committing to one token before the interpretation is settled, letting an early error propagate through the CoT.
• GRPO: Group Relative Policy Optimization; scores a group of rollouts per prompt against each other, with no process annotations.
• Gumbel reparameterization: adding Gumbel noise to log-probabilities to make sampling differentiable.
• Shannon entropy: here, the uncertainty of the next-token distribution, used as the model's confidence in its candidates.

Source

• arXiv 2609.38717v1
• PDF
• Hugging Face
• GitHub · https://github.com/rafiibnsultan/Soft_Spatial_Reasoning

Related papers

• Best of 22 VLMs hits 58.5% on AgentVQA as visual-state errors dominate
• LoRA on 33B MiniMax-H3 cuts missed entity-count changes from 34.7% to 21.3%
• Qwen3-VL-8B training lifts RewardData artifact Finding F1 from 3.38 to 35.42
• RTAformer gains 4.8 SGDET R@10 on transition frames over matched OED
• Composite rays cut held-out catadioptric error from 75.7° to 1.12°
• WorldStream: LLM planning plus deterministic code streams an endless 3D world that keeps your edits

All paper explainers