Evaluating Bounded Autonomy in Regulated Agentic AI: A Diagnostic Harness with Constitutional Rewards, Escalation Labels, and Runtime Governance
Dipankar Sarkar
cs.CL, cs.AI, cs.LG
2026-09-28
RegLLM makes when-to-escalate a trainable signal for regulated agents; two nominally identical Qwen2.5-3B DPO pilots swap metrics, exposing pilot-scale evaluation noise.
LLM agents are being pushed into regulated domains: financial compliance, healthcare advice, legal triage. The cost of an error there is a regulatory fine, a harmed client, or a missed obligation, not a regenerated reply. Neither dominant technique family covers the core decision. RL with verifiable rewards (RLVR) works on math and code because correctness is machine-checkable; regulatory advice has no such ground truth, since experts legitimately disagree about how a principle applies to a fact pattern. Guardrail systems are mature at blocking harmful actions from outside the agent, but they say nothing about the central question of a regulated workflow: should the agent answer this, or hand it to a qualified human? In deployed systems that call lives in hand-coded thresholds.
The pivot is a shouldescalate label on every task. The act-versus-defer decision becomes a verifiable training signal, scored into the reward like a correctly cited provision: measurable, trainable, auditable.
RegLLM is a diagnostic harness with a few layers:
Offline reference run (n=12, lexical baseline policy, heuristic judge):
| Metric | Base | Governance |
| Task success | 0.08 | 0.25 |
| Citation validity | 0.75 | 1.00 |
| Escalation recall | 0.00 | 0.67 |
| Unsafe-action rate | 0.33 | 0.08 |
On this weak baseline, which never defers on its own, the deterministic supervisor lifts escalation recall from 0 to 0.67, cuts the unsafe-action rate to about a quarter, and forces citation validity to 1.0.
Two GPU pilots (n=8, same seed, same eval split, same code commit, same hyperparameters, one rented 24GB-class card, Qwen2.5-3B LoRA/DPO):
| Config | Run A: success / recall | Run B: success / recall |
| RL-base | 0.25 / 1.00 | 0.12 / 0.50 |
| RL + answer-quality DPO | 0.12 / 0.50 | 0.25 / 1.00 |
| RL + escalation-aware DPO | not trained | 0.12 / 0.50 (no change) |
The numbers swap exactly: Run A's base matches Run B's adapter and vice versa. The same answer-quality adapter cuts recall from 1.0 to 0.5 in Run A and raises it from 0.5 to 1.0 in Run B. An escalation-aware variant, built specifically to fix the failure mode hypothesized in Run A, produces no measurable change in Run B. Citation validity also drifts from 0.50 to 0.38 between runs. Only the GPU host differed; container digests, package microversions, and CUDA determinism flags were not pinned, so floating-point ordering and kernel selection are candidate sources of the variance.
The runtime supervisor logged zero interventions across both GPU pilots: the model cites a provision on every answered turn, slipping past the no-citation gate, and trajectories that should have escalated never reach a terminal answer for the gate to inspect.
The useful output is not an adapter. Escalation labels turn bounded autonomy from a hand-tuned threshold into a trainable, auditable objective, which is the foundation regulated agents have been missing. The harder lesson: at n=8, 30 DPO steps, and 10 to 20 preference pairs, a budget common in agent pilots, configuration variance swallows the adapter effect the pilot was supposed to measure. A direction reported from a single smoke run is not evidence, in either direction. Teams evaluating agents should put multi-seed runs and larger eval sets in the budget.
The intended user is a team shipping an agent into a domain with a written rulebook, a way to flag out-of-scope queries, and a small open model to finetune. This is not a turnkey compliance product: the constitution, the corpus, and the escalation labels all remain the practitioner's work.
The author's own list is thorough: n=12 and n=8 are diagnostic-demo scale; single seed, single base model, single domain; escalation labels are author-generated from templates, with no expert annotation and no inter-rater protocol; grounding is a lexical-overlap proxy with a heuristic judge; runtime governance fired zero times in the GPU pilots; only the FCA instantiation exists.
Three gaps sit outside that list. The variance finding itself rests on two runs, so it is a methodological critique of pilot scale, not a verdict on DPO or escalation training. The answer-quality adapter used 10 preference pairs in Run A and 20 in Run B, so the cross-run comparison confounds pair count with host variance. And there is no sensitivity analysis for reward weights or the judge, so whether one constitution governing three roles survives a judge or weight change is untested.