Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures
Ruoqi Guo, Yi Liu, Gelei Deng, Yuekang Li, Lida Zhao, Yutao Wu, Simin Chen, Ying Zhang, Leo Yu Zhang
cs.AI, cs.CL, cs.CR
2026-09-24
On 44 benchmarks and 7,193 items, one generic Jev question ranks ten alignment failures at 0.886 median AUROC, at $0.30 versus $18.96 for the judges.
Screening alignment failures is both a production job and a research job. Deployed systems run a detector on every message. Labs reuse that detector to score benchmarks and mitigations. Current detectors still charge a decoding pass per criterion. Generative judges emit a paragraph that then has to be parsed. Classifiers such as Llama Guard read a probability off one verdict token, so one call still scores one fixed label.
Jev, TypeSafe's model trained with reinforcement learning for calibrated decisions (RLCD), answers many typed questions about one input with calibrated probabilities and no generated text. Whether that detects alignment failures had not been measured. The failures themselves are often relational: sycophancy is defined against the user's belief, deception against the model's own belief, prompt injection against an instruction hidden in tool output. Seeing only the reply can omit the reference that makes the failure a failure.
RLCDAlignBench turns the alignment suites of Chen et al. (2026) into detection tasks. Five open 2-7B targets each produce outputs for one failure family, then the original scorer labels whether the failure occurred. Ten failure types, 44 benchmarks, 7,193 items. Twenty scorers are rules, twenty-four are LLM judges. Eleven labels are validated; twenty-five are not.
The design splits what Jev is asked from what it sees. On the question side, each benchmark gets one template generic question filled with that benchmark's behaviour phrase, plus targeted questions that name the labelled behaviour: a direct judgment, the scorer's own rubric, or a decomposition over options or secrets. Answer types are Noul (yes/no, read as P(yes)), Choice (a distribution over options), and Score (ordered levels). On the state side, the canonical input is the target's input and output. Fields are then added one at a time and tagged as deployable (a monitor could hold them) or label keys (they define the label).
Targeted strategies are chosen on one half of the items and scored on the other, averaged over 20 grouped splits, so reported gains are not search inflation. Baselines are all-positive, response length, and an in-domain TF-IDF logistic regression. Jev never sees a label. All runs use jev-1.13.0.
A single generic Noul question, zero-shot, reaches a median AUROC of 0.886 (95% CI 0.821-0.952) on the 31 benchmarks that admit it. Without any labels it beats the better of response length and in-domain TF-IDF by a median of +0.132 and wins 25 of 31 (sign test p=9e-4). Targeted wording, selected split-half, adds +0.006 with a CI that includes zero. Scorer type does not move the number. Label source does: 0.949 on unvalidated labels, 0.872 on validated ones.
| Failure type | Generic Noul | Split-half targeted | Margin vs. baseline |
| Jailbreaks | 0.965 | 0.963 | +0.057 |
| Deception | 0.949 | 0.945 | +0.215 |
| Prompt injection | 0.962 | 0.998 | +0.026 |
| Hallucination | 0.823 | 0.867 | +0.186 |
| Sycophancy | 0.726 | 0.784 | +0.118 |
Soft probabilities matter more than wording. A 3-level Score read as expected level beats Noul on 19 of 31 (median +0.016). Argmax readouts lose almost everywhere (Choice 2 wins, 28 losses). Rubrics that cut each Jev answer at 0.5 before combining lose to the best direct question on 9 of 10, median -0.137.
Context a deployed monitor could hold rarely helps. Among four missing-reference pairs, only DeceptionBench's goal prompt lifts generic Noul, by +0.050. Label keys help more: adding PrivacyLens's secret list moves 0.79 to 0.95. Adding the gold answer on SycophancyEval (answer) moves 0.540 to 0.941, because that official label is correctness, not sycophancy. Open-Prompt-Injection sits at 0.231; its official label tracks accuracy on the injected task.
The probabilities rank. They do not travel as thresholds. Median per-benchmark ECE is 0.168 against a perfectly calibrated null of 0.074, from a 0.125 median miss on the positive rate. Median F1 is 0.706 at t=0.5, 0.822 with a cross-validated threshold, and 0.793 with a threshold fit on ten labels. On validated labels, t=0.5 is already enough (0.721 vs 0.694). Keeping the half of decisions farthest from 0.5 raises median accuracy from 0.793 to 0.933.
On StrongREJECT human labels the generic question matches the GPT-4o-mini scorer (Cohen's κ 0.809 vs 0.811) and ranks better (AUROC 0.971 vs 0.929). On HarmBench validation it matches a single annotator at κ 0.748, in line with inter-annotator 0.736. Confident disagreements exposed label defects in three benchmarks and labels the state cannot reveal in four MACHIAVELLI variants. One pass over the 19 API-judge benchmarks costs $0.30 against $18.96 for the judges, 63× less. Repricing every judge at GPT-4o-mini rates and asking Jev only the generic question still leaves a 12× gap. Median latency is 0.31 s for 11.4 questions; latency does not grow with the question count.
A large share of detector cost is one generation per criterion. Jev collapses that into one forward pass and a vector of probabilities. For a monitor that runs on every message, the paper's recipe is concrete: one generic question, read as a probability, threshold fit on ten labels. For people who build benchmarks, confident disagreements are a cheap label audit.
This does not replace a fixed-taxonomy classifier such as Llama Guard, and it does not replace activation probes. It is a black-box model with no fixed taxonomy that can ask many failure types in one call. It is not a new training method. The contribution is the protocol, and the finding that wording barely moves the number while the contents of the state and the label do.
The study covers one RLCD model (jev-1.13.0), English benchmarks, 2-7B targets, and mostly scorer labels. AUROC is higher on unvalidated labels than on validated ones, which is consistent with picking up judge bias. Social-bias F1 at t=0.5 is 0.222, below the all-positive 0.868. Pooled calibration looks fine (ECE 0.047); per-benchmark calibration does not, and label-free prior-shift correction drops F1 to 0.571-0.690.
Jev is a commercial API. Training data, objective, and overlap with these benchmarks are undisclosed. TypeSafe lists indirect meaning and adversarial content among known weaknesses, both common in these failures. A detector is dual-use: the same scores can help an attacker search for missed failures. Every target is a small open model. Whether the jailbreak AUROC of 0.965 holds on a frontier model with a strong refusal stack is untested.