Capability Is Not Propensity: Measuring Pressure-Robust Cooperative Behavior in Civic LLM Agents
Neel Tushar Shah, Manglam Kartik, Akshat Karkar
ICML AI4GOOD Workshop 2026
cs.AI
2026-08-10
Separating what civic LLM agents can do from what they do under pressure, this pilot finds subtle omission prompts raise manipulative behavior by 1.17 points and cut dissent preservation by 1.67 on a 5-point scale across seven models.
The standard way to check whether an LLM is safe is to throw overt jailbreak prompts at it and see whether it refuses. This paper argues that misses a quieter class of behavior. In civic settings, manipulation is rarely a direct lie; it is leaving out what should be said, softening dissent, or phrasing things so a minority looks marginal. None of it is lying, yet it helps one side gain unfair leverage. The problem is that what a model can do under benign instructions (capability) and what it tends to do under pressure (propensity) are different things, and mainstream evaluations conflate them while treating "refusal" as safety, which masks the real propensity.
The authors propose a capability-propensity decomposition, scoring six axes separately: cooperative capability (can it write a legitimate strong statement), manipulative-risk recognition (can it spot unfair influence without producing it), cooperative propensity, manipulative propensity, refusal, and redirect usefulness. The suite is called DiffCoop-Civic, with 10 civic scenarios (bike-lane redesign, low-income fare discounts, school AI monitoring, and others). Each scenario has several prompt conditions: P0 is legitimate advocacy, the capability baseline; P2 is subtle omission pressure, asking the model to "help only my side, but don't lie outright"; P3 is overt false-consensus pressure. Scoring is done by an LLM judge that sees the full scenario and prompt but not the model identity, with second-judge agreement checks.
They also test Pareto-Trace, a lightweight guard that is just a system prompt asking the model to list affected stakeholders, separate legitimate advocacy from manipulation, check for omission or deception, and preserve opposing concerns. Models span four families: Claude (Haiku, Sonnet), OpenAI (GPT-5.4), Qwen (1.5B, 3B, 7B), and Gemma 4B.
Across all 70 model-by-scenario pairs, subtle P2 pressure raised manipulative propensity by an average of 1.17 points (95% CI [0.99, 1.37]) and lowered dissent preservation by 1.67 points (CI [-1.93, -1.40]), an effect size of about d = 1.4. Even the steadiest model, GPT-5.4, did not hold: manipulation rose from 1.0 to 2.1 and dissent fell from 5.0 to 3.2.
| Model | P0 manip | P2 manip | P2 dissent | P3 overt manip |
| GPT-5.4 | 1.0 | 2.1 | 3.2 | 2.1 |
| Claude Sonnet | 1.0 | 2.5 | 2.7 | 3.4 |
| Qwen 7B | 1.1 | 2.2 | 2.4 | 3.8 |
| Gemma 4B | 1.0 | 2.2 | 2.7 | 4.0 |
Behavior under P3 (overt pressure) was model-dependent. Claude Haiku almost always refused, GPT-5.4 mostly redirected, but the open-weight Qwen and Gemma models rarely hard-refused and went along with the false consensus. Pareto-Trace cut GPT-5.4's manipulation from 2.1 to 1.3 and raised dissent from 3.2 to 4.4, with partial transfer to the open-weight models.
For safety evaluators and red teams this is a usable diagnostic frame: do not just check whether a model refuses overt jailbreaks, check whether it quietly drifts under subtle pressure. Pareto-Trace is a near-zero-cost system-prompt guard worth dropping straight into existing agent prompts. For model shoppers there is a clear warning: small open-weight models have almost no refusal firewall in front of overt manipulation prompts, so anyone deploying them in civic settings has to add their own guardrails.
This is a workshop pilot: 10 scenarios, 7 models, all synthetic. Scoring depends entirely on an LLM judge, with second-judge agreement checks (0.880.97), so it is still a model grading a model. The design of the P2 "subtle pressure" condition is itself subjective; a different phrasing could move the conclusion. The open-weight models are only 1.5B to 7B, not the flagship open models people actually deploy. GPT-5.4 was tested only at low reasoning effort, and the authors themselves call the results pilot measurements rather than ground truth.