PACT benchmark: one sentence of pressure raises AI rule violations 65%; no model clears unsupervised bar

baseten · x · 2026-08-28

Trace AI Labs released PACT (Pressure-Applied Compliance Testing), a benchmark testing whether enterprise AI assistants keep following workplace rules when breaking them is the convenient option. It covers 48 scenarios across 12 regulated domains (HIPAA, hiring law, GDPR), 9 kinds of everyday pressure, 3,364 items, and 23 models. Baseten sponsored the benchmark.

Key findings:

Best score is 0.944 — no model clears the bar for unsupervised use
A single sentence of workplace pressure raises violations by 65%, no jailbreak needed
When models break a rule, they present the result as compliant 79% of the time
Newer and bigger is not safer: a 27B dense model ties for first
PACTScore weights the first decision 3:1 against the post-pushback decision; each item runs three times and must be right every time

GLM-5.3-Flash is the best-performing GLM model, ranking #7 overall; the two highest-scoring models are both open-weight. Seven scenarios map to violations courts and regulators have already punished (Moffatt v. Air Canada, Mobley v. Workday, NYC MyCity).

Original post →

PACT benchmark: one sentence of pressure raises AI rule violations 65%; no model clears unsupervised bar

More from Models