WFM: Wiki Foundation Model for Complex Agentic Reasoning
Junnan Dong, Linhao Luo, Senlei Zhang, Gong Chen, Taian Guo, Yifei Yu, Rong Tao, Tao Guo, Qian-Wen Zhang, Siyu An, Ruizhi Qiao, Xing Sun
cs.AI
2026-09-16
WFM encodes entity graphs plus passages as an LLM Wiki, hits 89.6 Open accuracy on HotpotQA, and cuts distributed training from 2.40s to 0.23s per step.
Agents need persistent memory. Flat RAG cuts documents into disconnected chunks; GraphRAG compresses them into triples and throws away passage density. Teams have started writing knowledge as linked Markdown, an LLM Wiki: the paragraphs stay, and so do entity links. Encoding that dense hybrid with a classic graph model flattens attention into near-uniform weights and freezes gradients. Syncing boundary nodes across GPUs through CPU Gloo and Pickle then turns each training step into seconds of copy and serialization. Tencent Youtu Lab, with Monash, Hong Kong Baptist University, and Shenzhen University, stacks a Wiki schema, attentive aggregation, and an NCCL boundary exchange into one training path called WFM.
The Wiki Graph has two node types: entities and passages. Entity-entity edges keep typed relations; entity-passage edges are cross-layer hyper-edges, so a paragraph does not have to become another triple to enter the graph. Entities start from a learned lookup, passages from a pretrained language model, then a linear map puts both at the same width. A query first selects seed entities and passages; propagation runs only on that induced subgraph.
Relation-aware attention scores \(\pi(v,r,u)=wa^\top\tanh(Wr hu+er-Wr hv)\), softmaxes over neighbors, and aggregates. The update has two branches, sum and Hadamard product, each through LeakyReLU, so structure and text compete in one neighborhood. Retrieval sits in a self-reflection loop with budget B=4: each round adds TopK passages to a growing evidence pool, and the generator emits an answer, a follow-up query, and a stop flag. Easy questions exit early; the cap is four rounds, 2.58 on average in the default sweep.
Training uses three losses: TransE-style structure ranking, entity-passage InfoNCE, and a hinge that fires only when attention-logit variance falls below \(\epsilon\). A warm start freezes the graph encoder for alignment, then unfreezes everything. Defaults are three attentive layers, \(\epsilon=0.05\), \(\lambda2=0.1\). For distribution, partition boundary indices are computed offline, padded to a fixed shape, and exchanged with NCCL AllToAll on the GPU, skipping per-step CPU packing.
Answers come from DeepSeek V4 Flash, judgments from DeepSeek V4 Pro, embeddings from all-MiniLM-L6-v2, retrieval depth 20.
Multi-hop Open / Reject accuracy:
| Method | HotpotQA | 2Wiki | MuSiQue |
| Youtu-GraphRAG | 86.8 / 80.2 | 87.0 / 77.6 | 65.7 / 47.5 |
| WFM | 89.6 / 84.3 | 90.2 / 82.4 | 69.8 / 52.6 |
Gains versus Youtu-GraphRAG are larger in Reject (4.1, 4.8, 5.1 points), which is the setting that forbids parametric repair. Recall@20 is 93.20 on HotpotQA and 90.15 on 2Wiki, best in the table; MuSiQue is 75.24, 0.66 behind Youtu-GraphRAG's 75.90. On long memory, PersonaMem overall is 58.49 and RHELM 52.17, 8.39 and 5.37 above the strongest non-WFM baseline. Without reflection the scores are still 54.12 and 48.90. PersonaMem Recall@20 is 52.63 versus A-mem's 38.2.
Ablation: dropping passage nodes costs 7.48 average multi-hop Recall@20 and 7.68 memory accuracy; removing variance regularization or the warm start hurts both. Forcing all four reflection rounds recovers most quality at 1.58× cost. The NCCL path cuts a step from 2.40s to 0.23s, reported as a bit-exact 10.5×.
This is a trainable encoder plus a communication layer for Markdown Wiki memory, not a new agent philosophy. Three pieces travel: passages as first-class nodes, a variance floor on attention, and fixed-shape GPU boundary exchange. The HotpotQA lift over in-house Youtu-GraphRAG is a few points. The 2.40s to 0.23s step time is the part other graph-RAG stacks can steal.
Section 4.4 says newly completed table cells and diagnostic sweeps are constructed planning values and need measured runs before external use. Some headline numbers may still be plans, not measurements; cite that sentence with the scores. Generator and judge are both DeepSeek V4, so the numbers are not comparable to GPT-judged RAG tables. How the Wiki is built from raw text, and what happens when extraction is wrong, is barely described. PersonaMem recall uses an LLM judge. The PDF still carries a 2018 Woodstock conference placeholder, which reads as an internal preprint. The authors only sketch real-time dynamics and broader generalization; there is no production latency or cost.