EvoOntology: A Self-Evolving Ontology Layer for Data Agents
Meiduo Chong, Shaolei Zhang, Ju Fan, Xiaoyong Du
cs.AI, cs.CL, cs.DB
2026-09-14
EvoOntology serves a self-evolving ontology over MCP. Mean DDR Traj-Wise rises 17.8 points; a static semantic layer drops Claude-Sonnet-5 by 15.0.
Data agents answer natural-language tasks over tables, databases, and files. The data lives outside the agent; the agent only has generic tools such as SQL and file readers. Schema is unknown, names do not match, and grounding found on one trajectory is usually thrown away. That is the agent–data gap.
Blind probing does not scale on wide, heterogeneous sources. Stuffing a hand-built semantic layer into the prompt blows the context window and does not track how different backbones actually use the layer. This paper turns the middle layer into an ontology the agent can query, then lets that ontology edit itself from failure traces.
The ontology state has three layers. Content is a typed semantic graph: Terms, Mappings, Constraints, Evidence, plus semantic relations and structural references. Schema defines node fields and allowed relations. Tool exposes browse and resolve over MCP; only a short session manifest sits in the prompt, and detailed records are fetched on demand.
A builder agent never sees gold answers. It proposes candidates from recurrent entities, metrics, and operations in the training workload, probes the raw sources, and commits a candidate only if types, filters, and value distributions check out. Probe records stay as Evidence.
An evolution agent mines historical trajectories for recurrent failure signatures, tags each to Content, Tool, or Schema, and patches one level at a time. Parent and candidate are scored on the same validation split with identical decoding and interaction budgets; an update below margin τ is discarded. Six LLM backbones evolve independently from the same initial state.
Evaluation is reciprocal two-fold: 70% of one fold for construction and evolution, 30% for paired validation, freeze, test on the other fold, then swap. Held-out answers never flow back.
DDR-Bench 10-K, versus ReAct with no ontology:
| Backbone | Traj-Wise baseline | EvoOntology | Gain |
| GPT-5.5 | 64.2 | 90.9 | +26.7 |
| GPT-5.6-sol | 68.5 | 93.5 | +25.0 |
| Claude-Sonnet-5 | 72.5 | 81.3 | +8.8 |
| Claude-Opus-4.8 | 73.0 | 92.3 | +19.3 |
| DeepSeek-V4-Flash | 30.3 | 52.3 | +22.0 |
| Qwen3.5-Flash | 14.3 | 19.1 | +4.8 |
Mean Traj-Wise gain is +17.8. Pasting the builder's semantic layer into the prompt (Baseline+SL) is unstable: Claude-Sonnet-5 drops 15.0 Traj-Wise points. Retrievable episode memory lifts mean Traj-Wise from 69.5 to 75.8, still 13.7 behind EvoOntology's 89.5.
On BIRD under Oracle Knowledge, EX rises 7.4 points on average and VES 8.6; Claude-Opus-4.8 EX goes from 67.5 to 78.3. InsightBench's mean Overall gain is only 1.9, with DeepSeek-V4-Flash at +6.1, and short reference-style insights saturate quickly.
The initial ontology already adds 12.3 Traj-Wise points on DDR; evolution adds 7.7 more. BIRD EX moves +5.1 then +3.7. Removing the paired gate costs 11.2 Traj-Wise points, the largest ablation; removing level attribution costs 6.3. Tool-only evolution reaches 82.7 from 69.5; all three levels reach 89.5. Masking Mappings drops 13.4 points, Evidence 8.7. Pairwise Jaccard overlap of evolved Term sets stays at most 0.62, and cross-backbone serving drops at least 6.6 points.
Cost on DDR falls from 52.6K to 42.0K tokens per task, about 20% less, because trajectories shrink from 14.6 to 8.4 turns, which more than pays for the extra per-turn manifest.
A semantic layer should be queried, not pasted. For text-to-SQL and multi-source research, this is closer to a runnable system than another hand-written metrics file: MCP tools, probe grounding, failure attribution, and a paired gate. Each backbone wants its own evolved store; shipping one universal ontology across models loses points.
Do not expect large extra gains on short insight tasks that already match a reference.
Workload adaptation uses the other fold of the same benchmark, not a fresh domain. Sensitivity to τ and the number of evolution rounds is not swept in the main text. Insight scores are short and saturated, so evolution barely shows. Cross-backbone results justify per-model stores and also imply maintenance cost that grows with the number of backbones. Evolution compute is not billed separately from task tokens. Production databases whose schemas change daily are untested.