Asking for What Was Never Requested: Horizontal and Vertical Proactivity in Agents
Ido Levy, Asaf Yehudai, Segev Shlomov, Asaf Adi, Leshem Choshen
cs.AI, cs.CL
2026-09-29
Q&D trains an 8B questioner on retrieval outcomes. Equal-spend MuSiQue coverage rises 78% to 90%; it beats a 15× model on two of three suites and more than doubles retail success.
A tool-using agent typically handles what the user said out loud. Finishing the task often needs facts the user never named. In the retail example, a customer wants to change the size of boots in a pending order, gives a name and ZIP code, and cannot recall email or order ID. The same model, prompted, asks for those two fields sixteen times and never changes the boots. A proactive policy looks up the account from name and ZIP, follows it to the order, then to the size-8 variant, and completes the job.
Work on proactive agents mostly asks whether and when to act unasked. Agentic search either follows the chain the question already spells out, or trains only on final-answer correctness, so what each query pursues is neither measured nor rewarded. Evaluations can also treat asking more as asking better.
This paper scores a different axis: content. Horizontal proactivity pursues unstated needs the current state already makes nameable, such as the account at the start. Vertical proactivity pursues needs that become nameable only after earlier evidence arrives, such as the order ID inside that account.
A need graph, recovered mechanically from a multi-hop benchmark's own decomposition, lists required evidence and prerequisite edges. Depth is the longest prerequisite path above a need. A run is scored from its transcript against the graph, with no model judge. Breadth, the number of independent lines advanced, reads horizontal proactivity. Depth-weighted recall and the deepest need resolved read vertical. Stopping is read at each decision: halt once everything is found, keep asking while it is not. The policy never sees the graph.
Q&D splits the agent. A questioner asks one question or stops. A frozen drafter folds retrieved passages into a draft, so every state change traces to that question. Training forks a run at one step, samples eight candidate questions, continues each to the end, and ranks them by consequence: whether the run answered the task (this decides 6% of pairs), who assembled the required evidence sooner, then how much evidence each turn added. At unfinished states, asking ranks above stopping. There is no reward model and no judge.
The questioner is Qwen3-8B with LoRA, trained in three stages: imitation of good decisions, DPO on question pairs, then stop contrasts that prefer asking while work remains. Drafter and answerer stay GPT-OSS-120B in every arm, so only the questioner differs. Retrieval is BM25 over each task's released pool of 20 paragraphs (10 on 2Wiki), one call per question.
Held-out splits use 200 tasks per suite. Equal spend compares both policies at the lower question count, so asking more cannot pass for asking better.
| Suite | Prompted 8B coverage | Q&D | Depth-weighted recall |
| MuSiQue | 78.3% | 89.5% | 77.9% → 90.5% |
| StrategyQA | 78.4% | 85.5% | 51.2% → 56.6% |
| 2WikiMultiHopQA | 88.2% | 93.1% | 84.8% → 90.3% |
Against prompted GPT-OSS-120B, about 15× larger: coverage is 84.8% vs 77.8% on MuSiQue and 85.0% vs 80.7% on StrategyQA. On 2Wiki it trails, 88.9% vs 92.4%. The lead is largest where chains run deepest. On MuSiQue, Q&D allowed 4 calls recovers 4.1 points more evidence than the 120B model allowed 24. Per required need found, the trained questioner spends 11,610 tokens on MuSiQue against 19,354, about 40% less.
When each policy stops on its own, with a cap of eight calls, MuSiQue coverage is 91.2% versus 81.6% for the prompted 8B and 81.1% for the 120B, at 4.2 questions versus 5.9 and 5.3. Answer token F1 barely moves, from 41.1% to 43.6%. Extra evidence does not yet move answers.
Placed in a τ²-bench customer-service agent with no further training, retail success rises from 13% and 12% to 34% and 32%, with 1.3 and 1.4 fewer questions per dialogue. Against the 120B model at 17% and 19%, it leads by 17.0 and 12.7 points and needs 1.6 and 1.8 fewer customer follow-ups. Airline gains about 6.5 points over the same prompted 8B; no decided difference versus the 120B.
Replacing its questions with random questions from other tasks of the same benchmark drops MuSiQue coverage by 34.7 points. Hiding retrieved evidence from its state costs 8.4 to 8.7 points. Longer wording does not explain the gain: when the prompted model may spend as many question tokens, it still trails by 14.9 points on MuSiQue development. PAR2-RAG never beats plain prompting of its own model at equal spend. Prompted Claude Opus 5 leads Q&D by 2.9 points on MuSiQue at equal spend.
What an agent pursues can be scored without an LLM judge. The training signal is denser than final-answer reward. An 8B LoRA questioner can out-retrieve a prompted 15× model on deep chains when call budget is the constraint. Transfer to customer service without further training suggests the skill is what to ask and when to stop, not a QA template.
Do not expect answer metrics to follow. Training teaches what to ask more readily than when to stop, and coverage gains have not produced a detectable F1 lift. This is a retrieval-policy improvement plus a measurement tool.
The two reported seeds share one supervised checkpoint and one pair export, so intervals do not cover that source of variance. No checkpoint met the pre-committed selection rule. Training labels and the primary metric share the need graphs; answers decide only 6% of pairs. Released retrieval pools are small enough that most prerequisite edges do not actually gate retrieval, so the vertical result is depth in a decomposition, not a hard gate where A must be found before B can be searched. On MuSiQue and StrategyQA each task has one line of inquiry, so breadth is barely an independent axis. Every user-facing number comes from a simulated customer, on 25 retail and 17 airline tasks. Q&D is one round of off-policy training, not compared with on-policy RL, and the length control was not run held-out.
In the Figure 5 illustration, strict answer match is zero for all four runs. Coverage and getting the answer right are different outcomes.