Iris: Climbing to the Search Frontier
Ziyuan Liu, Hengqi Liu, Zichuan Wang, Yang Qin, Jiachen Liang, Xu Chu, Shaowei Chen, Yuantao Gu, Mu Chuan
cs.AI
2026-09-04
AllSpark reverse-builds multi-hop tasks from web graphs and alternates SFT with live-search RL. With CM, Iris-mini/pro score 82.2/88.6 on BrowseComp.
Leaderboards for search agents have moved fast, but a reported score often mixes the policy with the inference harness. Long-horizon search fills the context window; systems then summarize, prune, or wipe history and keep going. Those context-management (CM) tricks can add more than ten points. Publishing only the managed number credits the wrapper to the model.
AllSpark ships a full recipe (task synthesis, trajectory filtering, alternating SFT and live-search RL) and reports every benchmark with and without CM. Iris-mini is Qwen3.6-35B-A3B; Iris-pro is Qwen3.5-397B-A17B. Both are single ReAct agents with search and scrape only: no sub-agents, no test-time verification.
Questions are reverse-built from a web hyperlink graph. A seed page expands into a local subgraph, which is distilled into an entity graph; a question is written over a path of at least N hops. Every non-answer entity is rewritten as a descriptive reference with its name and aliases stripped, so string matching cannot skip the intended chain. A pair is kept only if a reference model fails closed-book and succeeds once the evidence graph is supplied.
A teacher then rolls ReAct trajectories against live search. Coarse filters keep trajectories that are correct, non-degenerate (sliding-window compression catches loops), and deep enough in tool calls. Fine filtering lets an LLM mask at most 10% of assistant turns; masked turns stay in context but drop out of the loss.
RL uses a group-relative policy gradient on live search. Over-long rollouts are interrupted at the request level and resumed from a committed prefix, with truncated importance sampling correcting splices across weight versions. The reward judge and page summarizer run as in-cluster FP8 copies of Qwen3.5-397B-A17B. SFT-RL climbing then feeds the shortest successful, sufficiently deep rollouts from queries whose group pass rate sits in (0, 1/2] back into the next SFT round, so difficulty tracks the current policy.
The headline numbers use discard-all: when context fills, wipe history and restart the same question (the DeepSeek-V3.2 recipe).
| Model | BrowseComp | BrowseComp-ZH | DeepSearchQA | HLE |
| XYZ-Aquila-mini 35B | 78.8 | 82.9 | 89.5 | 51.1 |
| Iris-mini 35B | 82.2 | 84.8 | 86.9 | 52.3 |
| XYZ-Aquila-pro 397B | 84.8 | 85.1 | 92.5 | 53.3 |
| Iris-pro 397B | 88.6 | 85.1 | 92.9 | 56.4 |
Without CM, Iris-mini still scores 64.7 / 72.3 on BrowseComp / BrowseComp-ZH, above FORT-Searcher's 55.9 / 62.1. Discard-all then adds 17.5 points on BrowseComp to 82.2; discard-all plus retry reaches 85.9. Iris-pro starts at 72.6 / 76.8 without CM, 88.6 with discard-all, and 90.3 with retry.
CM pays most on BrowseComp and least on HLE. Iris-mini's unmanaged HLE baseline is 43.2 and rises by about 9 points, because the bottleneck is expert reasoning, not a full context window. On BrowseComp-ZH three configurations all land on 85.1 (246 of 289), and the appendix flags a ground-truth clash with the source material.
The useful artifact is a search-agent recipe that separates data, training, and evaluation, and that publishes the unmanaged scores. A 35B Iris-mini already sits near Kimi-K2.6 (83.2) and DeepSeek-V4-Pro (83.4) on BrowseComp. Anyone who only reads CM-on tables will over-credit the policy.
Weights and the recipe are planned for release. Search data and search teachers also transferred to general tool use and office tasks; the report does not give those numbers.
A gap remains versus the strongest closed or heavy-compute systems: Kimi-K3 at 91.2 BrowseComp, Apodex-1.0-H at 90.3, Claude Fable 5 at 64.5 HLE. Retry can still lift the score at the cost of a full extra search, and the authors refuse to treat it as the primary report. Several climbing hyperparameters are deferred to a later write-up. Single-sample pass@1, an LLM judge, and noisy BrowseComp-ZH labels all say that a few-point gap at a given size should be read conservatively.