Unequal histories: frozen LimiX-2 moves the 50-benchmark mean from 78.80% to 84.92%

Decoupled Predictive Decision Making: A Shared Conditional Predictor for Agents

Lang Mo, Xingxuan Zhang, Qi Wang, Mingchao Hao, Hao Yuan, Chunjing Gan, Wang Lu, Peng Cui

cs.AI

2026-11-02

DPDM freezes the LLM as encoder and one LimiX-2 as predictor. Qwen3.5-27B mean accuracy on 50 benchmarks goes from 78.80% few-shot to 84.92%, and 82.33% with matched histories.

What problem this solves

Whether an agent should call a tool, which candidate it should prefer, and whether a reasoning step is valid all depend on relationships that show up in deployment feedback. The task text often leaves those relationships open. The same action can flip from right to wrong once the feedback changes.

Typical systems tie three jobs to one language model: reading the problem, inducing a relationship from history, and producing the answer. Few-shot prompting stuffs past cases and the current query into a single context. ExpeL distills experience and retrieves it, Rememberer updates an external memory with reinforcement learning, and DPT predicts actions from interaction context. The inference step still sits inside the language model or a task-specific module, so it cannot be swapped on its own. Two kinds of error get mixed: facts the history never contained, and facts the learner failed to use.

Method

DPDM (Decoupled Predictive Decision Making) treats a decision as conditional prediction. A task spec fixes which fields are visible, what the target means, and which candidates are legal. History is a set of records whose targets are already observed. The current decision is a query whose target is unknown.

The language model is only a semantic encoder. It embeds each record together with the candidate descriptions. The conditional predictor is LimiX-2, a model pretrained on structured data and frozen for every experiment here. It reads those embeddings plus historical labels supplied separately, and returns a probability for each candidate. The decision rule picks the maximum. Tool applicability, candidate preference, and step correctness are different targets on one interface: represented history, historical labels, current query.

Three choices separate this from few-shot prompting. Labels stay out of the encoding prompt, so the representation step cannot copy the answer. Records are encoded one by one, so history is not capped by a single context window. On FaithDial that means 8,192 records. Neural weights stay fixed. New feedback is appended to the history, and a feature adapter is refit on the updated inputs. All five task categories, and the Qwen3.5 and Llama-3.1 encoders, share one LimiX-2 checkpoint.

The setup is a table that gains rows. The language model turns text into columns. LimiX-2 predicts the label of a new row from the rows already there. Neither model is trained again.

Results

The evaluation covers 50 benchmarks and 82 task configurations in five groups: agent tasks, knowledge QA, evidence consistency, safety and reliability, and language understanding. Configurations are averaged inside each benchmark, then benchmarks are weighted equally. Baselines are zero-shot and few-shot prompting of the same language model, plus the Jev API v1.13.0 decision service. DPDM numbers are means over three sampled histories.

EncoderZero-shotFew-shotDPDM
Qwen3.5-4B64.56%60.51%80.78% ± 0.11
Qwen3.5-9B69.92%65.03%82.27% ± 0.15
Qwen3.5-27B74.02%78.80%84.92% ± 0.14
Llama-3.1-8B53.49%41.87%77.57% ± 0.13
Jev API75.94%79.30%not run

In that table, DPDM uses the longest history each method finished under its evaluation budget, so it can see more rows than few-shot prompting. With Qwen3.5-27B the mean moves from 78.80% to 84.92%, with gains on 46 of 50 datasets. Both smaller Qwen encoders improve on all 50, and Llama-3.1-8B on 49 of 50. Few-shot prompting scores below zero-shot for the 4B, 9B, and Llama encoders. Stuffing history into a smaller context hurts in this setup.

When the records, their order, and the queries are identical, the 27B gap shrinks to 82.33% versus 78.80%, an absolute 3.53 points. The matched scores are 78.37% versus 60.51% for 4B, 80.08% versus 65.03% for 9B, and 73.83% versus 41.87% for Llama-3.1-8B.

On agent tasks, under each method's longest history, Qwen3.5-27B moves from 76.41% to 85.18%, 9B from 64.30% to 83.16%, and Llama from 40.77% to 79.73%. The large tool-use gains use those unequal budgets too. BFCL goes from 42.59% to 76.85% (+34.26 points) and ToolSandbox from 68.04% to 96.91% (+28.87 points). CodeCritic Code goes from 76.82% to 90.85%, and QA from 65.85% to 76.62%. The main text does not break those tool tasks out on matched histories. In Figure 5, few-shot prompting stops at 64 or 128 records and Jev at 16 or 32, while the DPDM curves keep climbing. On FaithDial, DPDM with 8,192 records reaches 83.53%; Qwen3.5-27B few-shot with 512 records reaches 54.70%.

On 21 shared benchmarks, accuracy rises at every step as history grows from 32 to 8,192 records, with both networks frozen. Qwen3.5-27B moves from 75.48% to 84.87%, 4B from 69.78% to 80.98%, 9B from 72.65% to 82.33%, and Llama from 63.81% to 78.74%.

Skewed labels pull few-shot prompting toward the majority class. On five BIG-Bench Mistake tasks and all 7,032 query steps, matched histories hold 128 to 1,024 records, of which 66.41% to 95.31% are correct steps. Mean balanced accuracy rises from 59.86% to 74.37% (+14.51 points). The task mean rises on all five tasks, and 18 of 20 encoder-task pairs rise. On object tracking, both methods see the same 689 historical steps (77.36% correct) and 462 queries. Qwen3.5-27B few-shot reaches 69.07% balanced accuracy and 84.85% ordinary accuracy, rejecting 42 of 104 incorrect steps. DPDM reaches 92.62% and 94.37%, rejects 93 of those 104, and accepts 95.81% of the correct steps.

On six tasks, 128 historical records, and 737 queries, original pairings score 80.71% ± 0.41. Permuting historical labels drops this to 50.13% ± 0.80, and always emitting the most frequent label scores 50.52% ± 1.50. At the same 128 rows, repeating 16 records eight times scores 64.79% ± 2.02, while 128 distinct records stay at 80.71%, a 15.92 point gap. Across eight tasks and 18,759 queries, at a history ceiling of 512, removing semantic features drops accuracy from 80.42% ± 0.33 to 63.95% ± 0.31, and permuting labels drops it to 48.78% ± 1.99.

On five tasks with matched histories and 256 queries each, full-pipeline GPU time, including history preparation, averages 19.7% of cached few-shot prompting, 80.3% lower. Encoded history is reused across later queries.

Why it matters

The encoder and the predictor can be improved separately. A stronger Qwen still pairs with the same frozen LimiX-2. A stronger conditional predictor can move tool judgments, preferences, and step assessments together, without a new policy network per role. What changes at deployment is the history table. The weights stay frozen.

If the target was never observed, the table has nothing to fit. Tool success and step correctness have to be logged before they can be learned.

On the strongest encoder, the matched-history gain is 3.53 points. That is an incremental improvement. The big numbers come from two other places: few-shot prompting already falls below zero-shot for the smaller models, and DPDM can keep histories longer than one prompt. The abstract's 84.92% and the 34.26 point BFCL gain include the benefit of a longer history.

Limitations

The main table and the tool-task jumps use each method's longest finished history. A historical record costs a different amount in the two architectures, and DPDM usually ingests more of them. The matched-evidence comparison is the overall 82.33% versus 78.80%. Tool tasks have no matched-history breakdown. In Figure 5, Jev is given 16 or 32 records and plotted next to much longer DPDM curves.

The introduction names ExpeL, Rememberer, Algorithm Distillation, DPT, PreDeToR, and SayCan. The experiments compare only against prompting and the Jev API. Whether the shared predictor beats those experience-based methods is not measured.

The decision is an argmax over a given candidate set. Open-ended generation of the next action is outside the evaluation. The five groups include knowledge QA and language understanding. AG News, TREC, and FaithDial look closer to tabular prediction on embeddings. The agent story rests mainly on tool and step assessment, and those results use unequal history budgets.

Only one conditional-predictor checkpoint, LimiX-2, is tested. Few-shot prompting underperforms zero-shot for the 4B, 9B, and Llama encoders, so part of the gap on smaller models is a weak baseline. The representation network is not updated at deployment. Only the feature adapter is refit.

Terms

Source

Related papers

All paper explainers