Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
Feiyu Gavin Zhu, Xiaoyu Zhu, Jiqi Yang, Rui Yang, Arnab Kumar Mondal, Yancheng Wang, Xinke Deng, Jean Oh, Reid Simmons, Joerg Liebelt, Xiang Kong, Zhongyu Jiang
cs.LG, cs.AI
2026-10-01
Apple's SSR turns per-turn agent reasoning into selection among six fixed candidates scored in parallel; a 4B model matches SOTA (61.37% avg) with over 90% lower reasoning latency.
ReAct-style multimodal agents generate a freeform thinking block before every action, then decide which tool to call with which arguments. For big models this is a modest cost; for a 2B or 4B model it is expensive and often wasted, since limited reasoning capacity produces long deliberations that add little guidance to the action while every token costs sequential decoding time. The paper's own measurement is blunt: after GRPO training, a 4B model with freeform thinking averages 58.65% success across seven benchmarks, slightly below the 58.91% of the same agent with thinking disabled. At this scale the thinking block does not pay for itself.
Existing efficiency work, such as Chain-of-Draft prompting and RL rewards for shorter traces, compresses the text but keeps the autoregressive dependency between tokens. This Apple and CMU paper changes the representation instead: in multimodal search the high-level rationale recurs across questions. 'The entity is recognized but the requested fact is missing, so search the entity name plus the attribute' works whether the question is about a conference venue or the founding year of LEGO. The reasoning space does not need to span all of natural language.
SSR (Selection-based Structured Reasoning) replaces each turn's open-ended reasoning with selection from a fixed library of six hand-written natural-language candidates, unchanged from training through evaluation and shared across all seven benchmarks. They cover the generic situations a search agent meets: reverse-image search when the entity is unrecognized, text lookup when it is recognized but the fact is missing, zooming into a small region, answering from gathered evidence, plus answering from the image or from general knowledge.
Selection uses the model itself as the classifier, with no auxiliary head:
The speedup comes from parallelism. Candidate texts are fixed in advance, so teacher-forced prefill computes all token likelihoods of one candidate in a single batched forward pass, and all six candidates are scored concurrently over a shared history KV cache. During reasoning the model generates no new token at all. On the paper's worked example, freeform thinking costs 156 tokens and 1821.92 ms; the SSR selection covers 31 tokens in 32.66 ms.
The design separates what kind of step to take from how to execute it. Candidates state intent; queries, coordinates, and answers stay in action generation, so one candidate serves many contexts. An ablation backs this up: scoring candidate indices instead of full text drops average success from 61.37% to 58.98%, so selection needs the semantics of the candidates.
Training plugs into existing objectives. SFT combines a categorical selection loss with the action-token loss. In GRPO, the token-level importance ratios for reasoning collapse into one categorical ratio while action tokens keep theirs, and because reasoning is structurally confined to the library, RL trains from the base model without the SFT warm-up these pipelines usually need. GSPO and SAPO variants reuse the same components.
Seven benchmarks (MMSearch, HR-MMSearch, FVQA-test, InfoSeek, SimpleVQA, LiveVQA, MAT-Search) with Qwen3-VL 2B and 4B backbones:
| Comparison | Metric | Result |
| SSR 4B (GRPO) vs TAPO+GSPO 4B | Average success rate | 61.37% vs 61.25%, best in the 4B group |
| SSR vs freeform (GRPO 4B) | Reasoning latency per turn | 0.896s to 0.061s (-93.2%) |
| SSR vs freeform (GRPO 4B) | Model latency per question | 5.456s to 2.514s (-53.9%) |
| SSR vs freeform (GRPO 4B) | Effective reasoning throughput | 72.7 to 3029.9 toks/s |
| SSR vs baselines (profiling subset) | Success / p95 reasoning latency | 58.36% (highest) / 0.071s vs 0.798-1.892s |
Against the four efficient-reasoning baselines (Chain-of-Draft, Sketch-of-Thought, efficiency-reward RL, Probe & Prefill), SSR is at least about 7x faster on the reasoning span and also posts the highest success rate. Success is not flat across objectives: GRPO gains 4.6% at 4B and 2.6% at 2B, SFT is unchanged, GSPO loses 3.1% and SAPO 1.3%; per-question latency falls 28.7-53.9%. The library-size ablation is telling: growing from one generic candidate (text: 'I am a helpful assistant') to six raises average success monotonically from 37.24% to 61.37%, and the step from one to two candidates alone adds 17.27 points. Selection tracks behavior: in 99.9% of turns the generated action type matches the candidate's expected tool. The 2B agent averages 51.26%, close to the zero-shot 8B model at 51.33%.
Latency is also predictable. SSR's p95 reasoning time is 0.071s, about 10 ms above its mean, because scoring a fixed candidate set is constant work; freeform lengths vary, and baseline p95 values reach 1.892s. For online and on-device use, that consistency matters as much as the mean.
For teams building on-device or cost-sensitive agents, this is a directly reusable recipe: in search-style tasks, a small model's freeform thinking can be swapped for a tiny fixed selection space with no accuracy loss and a roughly 90% cut in reasoning latency, running on the prefill path of an existing engine (SGLang here) with no new components and no warm-up requirement. The method is orthogonal to the training objective, so it ports across GRPO/GSPO/SAPO/SFT pipelines. The paper also leaves an uncomfortable data point: at 4B, freeform thinking on average lost to no thinking at all, which suggests the problem is how deliberation is represented, not how much of it there is.
The fit is best where reasoning recurs and each step amounts to choosing the next action type. Tasks with genuinely open-ended reasoning needs are outside what the paper tests.
The authors acknowledge that GSPO and SAPO lose 3.1% and 1.3% success respectively, attributing this to the detached-competitor approximation of the selector gradient going stale when rollout batches are reused across updates. The efficiency gain is not free under every objective.
Open questions remain. The six candidates are hand-written for search; nothing here covers how to build or size a library for math, code, or open-ended planning, and automatic library construction is listed as future work. The 90%+ figure covers only the thinking span: end-to-end model latency per question drops 28-54%, and environment-side tool execution is excluded from measurement entirely. Parallel scoring saves latency, not FLOPs: every candidate's prefill is computed, and the paper's own accounting says generating 100 freeform tokens on an RTX PRO 6000 costs about the same time as processing a 15,200-token library, so a larger library erodes the margin. Latency profiling samples only 100 questions per benchmark, non-multiple-choice answers are scored by a Qwen3-VL-32B judge, and the SFT trajectories were synthesized by Gemini 3.1 Pro without separate validation of that pipeline.