Neos: Alibaba's 9B multimodal search model closes to within 2.4 points of Gemini-3.1-Pro

量子位 · wechat · 2026-11-14

Alibaba's Accio team, Fudan University, and Shanghai Innovation Institute propose Neos, a unified training framework for multimodal deep search combining data construction, process rewards, and agent-environment co-evolution — enabling agents to keep gathering and verifying visual evidence mid-search (e.g., counting knobs on a record machine starting from a band photo).

Three components

Data construction: multi-hop tasks from knowledge graphs with visual checkpoints; blind image verification, entity-leak-removing question rewrites, and four-stage filtering to eliminate text-only/short-budget shortcuts.
Credit assignment: successful trajectories keep group-relative advantages for diverse exploration; failed trajectories use verified references to pinpoint key mistakes with decaying downstream penalties, while rewarding effective exploration and self-correction.
MEI co-evolution: a trainable Multimodal Evidence Interface filters/compresses web and image results, alternating with policy training (RL for policy, SFT from live interactions for MEI).

Results

Neos-9B/35B average 60.7/64.7 across ten benchmarks, +17.4/+15.5 over their workflow baselines; Neos-9B beats DeepVoyager-VL-30B-A3B (58.6).
Neos-35B trails Gemini-3.1-Pro workflows by just 2.4 points; ties GPT-5.6-Sol at 66.4 on BrowseComp-VL.
Ablations validate each component; tool calls per trajectory dropped from 29.7 to 20.2 with more weight on reading sources and verifying visuals.

Code: github.com/Accio-Lab/Neos

Original post →

Neos: Alibaba's 9B multimodal search model closes to within 2.4 points of Gemini-3.1-Pro

More from Models