Contestable Agents for Data Governance: Recording Human Stewardship of AI-Generated Metadata Across Structured and Unstructured Systems
Humza Akhtar, Brian Freeman
cs.AI, cs.DB, cs.HC, cs.MA
2026-10-23
Contestable governance for AI metadata: specialist agents produce scored, cited arguments; a human steward contests them before commit; every edit is stored as auditable data.
Industrial operational knowledge lives in two places: structured records (telemetry, point lists, transactions) and unstructured ones (manuals, service notes, inspection reports). Governance programs manage them separately, and the work of classifying and linking them falls on a handful of domain experts. Commercial HVAC is the clearest case: a connected air handler exposes hundreds of points, one vendor calls a supply-air sensor AHU3SAT and another SupplyAirTemp03, and until someone maps those names onto a standard ontology like Brick Schema, energy reports and fault-detection rules cannot use them. Experts are scarce, so new equipment models pile up unclassified.
LLM agents change the economics: reading manuals, comparing past decisions, and proposing classifications can be automated. The problems follow directly. An agent that writes straight to the catalog propagates one wrong class across every unit of the same model. Human reviewers approve with a single click, a documented rubber-stamping pattern, and the record shows only that someone approved, not what they checked. When a reviewer overrules an agent, the correction stays in an email the agent never sees and the mistake repeats. The authors, from MongoDB and HVAC maker Trane Technologies, argue that contestability belongs in the data layer.
Seven requirements anchor the paper: proposals cite their sources; they can be contested before commit; decisions are attributable to an agent, model version, or named reviewer; history is append-only; full lineage is retrievable in one query; verified decisions pass a gate before becoming precedent; reviewer interventions are stored as measurable data.
The full workflow has four phases:
Scoring uses a QBAF (quantitative bipolar argumentation framework): arguments support or attack each other with numerical strengths, iterated toward a fixed point; the reference implementation uses edge weight α=0.3 and 10 iterations. A versioned domain glossary handles acronym ambiguity, with each entry carrying equipment and revision scope, a source passage, and an owner; DISCH means mixed air on one machine and discharge on another, and a single global expansion would hide exactly the conflict a reviewer needs to see. Triage routes high-confidence proposals through a lighter review and reserves full argumentation for uncertain ones; the threshold itself is a governance decision with a named owner.
Both use cases are synthetic and the paper reports no classification accuracy; the numbers show how scoring responds to human intervention.
Use case 1, HVAC point classification, ships as an open reference implementation (CrewAI, GPT-4o, Voyage AI embeddings, MongoDB). The synthetic site has 247 points, 18 flagged ambiguous; under triage, 229 get lighter evidence-based review and 18 get full argumentation, all reviewed before commit. On the demo point AHU-1-DISCH-T, DISCH means mixed air on EQ-7700 air handlers but supply air on the same series' fan coil units:
| Case | Metric | Value |
| Point classification | Margins before intervention | discharge 0.045 / mixed 0.044 |
| Point classification | After rejecting the name-only challenge | mixed 0.821 |
| Point classification | After adding an argument citing manual §4.2 | mixed 1.758 (committed) |
| Warranty decision | Sensor / Linkage margins before review | 0.907 / 0.839 |
| Warranty decision | After rejecting S3, adding S6 | Sensor -1.016, Linkage 0.839 unchanged |
In the warranty case, the steward notices the service bulletin covers only serial suffixes 1000–1999 while this unit is 2407, rejects the argument built on the bulletin title, and adds one noting the sensor swap did not clear the fault; the system selects damper linkage failure. Stability checks: max iteration-to-iteration difference 3.20×10⁻⁷, per-argument residual bound 1.37×10⁻⁷, and rerunning at α=0.2 and 0.4 preserves both winners. The triage section also cites someone else's 100-task benchmark where similar routing cut calls to a strong model by 72.7% at 95% task success; the paper states plainly those figures do not measure this workflow.
Any team letting agents write metadata hits the same wall: errors spread through the catalog, human review leaves no evidence, and corrections never reach the agent. This design turns what the human checked and changed into queryable data; one aggregation returns a decision's full lineage, and operational questions such as which agent's arguments get rejected most, or how long review takes for a new model, become statistics. The argument graph gives review useful granularity, since a steward challenges individual arguments rather than a final label, and the next unit of the same model inherits the correction through precedent. To be clear, this is a pattern paper plus a reference implementation, not an evaluation: QBAF and DF-QuAD are existing argumentation machinery, and the contribution is applying them to data governance with a storage model and gates.
The paper's own list is long. Base scores are assigned by the agent that writes the argument, remain uncalibrated, and can vary between runs; in the worked example they are simply author-picked. There is no measured accuracy, review time, or erroneous-commit rate. The reference implementation runs the full workflow on a single point, overwrites arguments on a second edit, never writes verified decisions back to its precedent library, and lacks triage, append-only history, and the promotion gate, all of which remain proposals. The scoring rule has not been checked against published axioms; an isolated argument's base score maps to 0.50–0.73 after the logistic transform instead of being preserved.
Three more spots invite skepticism. S1 and S2 both derive from the same claim record, so correlated evidence can inflate a margin without independent support; the paper names the problem but offers no fix. Brick 1.4.2 declares the supply-air and discharge-air sensor classes equivalent, so the three labels are two ontology choices, yet the implementation scores all three. Every correction requires a steward, which delays even low-risk edits; the sketched automatic path is unevaluated.