ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation
Aurélien Pellet, Julien Perez, Marie Puren
LRE
cs.AI
2026-09-15
ORDER routes sources with QRe and splits quota via UMS. On 438 multi-hop 1887 French questions, Recall@3 rises 37.7 to 53.7 and answer accuracy 31.0 to 48.1.
Expert-domain RAG is rarely one index for every question. Historians do not ask interchangeable questions: some need two newspapers read against each other, some must enter the parliamentary record, and chunk size, metadata filters, and source choice all move with intent. A single segmentation and a single retriever let the largest collection monopolize the budget, or pour unrelated debate chunks into the top-k.
Recent work revisits chunking, yet still often tunes segmentation, retrieval, and the query on separate tracks. ORDER splits the job: discover strategies offline from documents and question clusters, then route a new query onto the matching configuration at trivial cost.
The corpus is three 1887 sources from HistoriQA-ThirdRepublic: the parliamentary record Les Débats and two dailies, Le Gaulois and L'Intransigeant. 875 historian-validated multi-hop questions split 437/438 train/test. All embeddings are Cohere embed-v4.0 in three Chroma collections. Cohere Command-A both writes answers from the top-10 hits and judges them against gold documents.
Three mechanisms share the same offline-discover, online-route pattern.
Task-conditioned chunking touches only Les Débats; newspapers stay at article granularity. Fourteen segmenters group into baseline windows, speaker turns, procedural markers, and agenda topics. Training questions go through UMAP to 10 dimensions and HDBSCAN. Each cluster, noise included, picks the chunker with the best Coverage@3 and stores that index. At inference the query lands in the same UMAP space, takes the nearest centroid, and reads that index.
Task-conditioned metadata reuses the clusters and, on the baseline 10000hierarchical index, picks a filter, a rerank, or both.
For structured retrieval, QRe is a 3-class logistic regression over the query embedding, with labels {G+I, I+D, G+D}. A routing map collapses this to a binary call: whether Les Débats joins the two newspapers. UMS then splits the remaining budget evenly across active collections so the large archive cannot eat every slot.
Per query online: one embedding, one linear router, at most |C| ANN lookups, one generation. No LLM pass over the corpus at index time.
Raw 3-class accuracy is 80.1%, with almost all errors between the two Débats-including classes, which retrieve the same collections. On the binary decision that actually matters, accuracy is 98.9%. Only 2 of 287 Débats questions are sent to newspapers-only; only 3 of 151 newspaper questions pick up an extra parliamentary collection.
| System | Recall@3 | Recall@10 | Cross-paper@3 | Paper↔Débats@3 |
| Naive Retriever | 37.7 | 56.2 | 16.6 | 48.8 |
| BM25 | 23.5 | 39.5 | 4.6 | 33.3 |
| HippoRAGv2 | 31.7 | 53.0 | 11.6 | 42.7 |
| LinearRAG | 18.7 | 40.1 | 6.2 | 25.1 |
| ORDER QRe | 48.7 | 70.4 | 48.7 | 48.8 |
| ORDER UMS | 52.1 | 70.3 | 44.4 | 56.1 |
| ORDER QRe+UMS | 53.7 | 73.1 | 49.0 | 56.1 |
On cross-newspaper questions the naive retriever is flooded by debate chunks; QRe lifts @3 from 16.6 to 48.7. On newspaper-Débats questions UMS's quota lifts @3 from 48.8 to 56.1. Together they win every overall cutoff. The same router on BM25 moves Recall@5 from 30.7 to 57.0. End-to-end accuracy moves from 31.0 to 48.1. Absolute numbers stay modest because the questions are long, often interpretive, and the judge scores gold documents rather than short strings.
Chunking helps the naive retriever and is mostly absorbed once QRe+UMS is on. Metadata still helps under UMS by reinforcing the parliamentary quota.
On multi-collection archives that are size-imbalanced and OCR-noisy, the failure is often between collections, not inside a document's split. ORDER pulls "which libraries, and how many slots each" out of the generation loop into one classifier and one even split. Legal, administrative, and multi-KB support stacks share the preconditions: collection sizes that drown each other, and source need that is predictable from the question text. Both can be measured on a small labeled sample before anyone deploys.
Transfer is argued, not shown: one slice of 1887 French, with cluster maps and the router bound to that corpus. Evaluation is multi-hop only; single-hop traffic is untested. Questions were drafted by Command-A from paired passages and then historian-filtered, which may favor lexical overlap and partly explain BM25's collapse. Every dense index and the router features use embed-v4.0. End-to-end uses the same model as generator and judge, one split, no human final pass. UMS trades precision for coverage and the paper does not separate added noise from answer gain. The router fits a few hundred labels, so small clusters will jitter.