60-model stress test: whether LLMs fold to user pressure tracks model age, how they fold tracks the vendor
ceciletamura · x · 2026-10-26
arXiv paper "Conduct Under Pressure" by Tapan Parikh (Cornell Tech) sends frozen multi-turn scenes — users insisting, begging, flattering, or grieving — to 60 models from 13 vendors.
Key findings:
- Whether a model folds tracks recency: fold rate correlates with a public capability index at Spearman -0.64, with little vendor effect.
- How it holds tracks the vendor: 6 of 17 manner codes sort by vendor at permutation p≤0.001, corrected across the codebook.
- On labeling: six LLM coders from three vendors outperform three human coders on consistency (Krippendorff's alpha 0.66 vs 0.46), agree with the codebook author at kappa 0.84-0.91 on trajectory, and match an adjudicated human reference at 0.83 — but blind machine readings can't predict which codes a second reader would apply the same way.
More from Models
- Editing just 2 neurons instantly "flips" Qwen 3.5 4B, netizens joke — jaivinwylde · 2026-10-26
- ChatGPT Pro user with 95% quota left hit with "capabilities reduced" notice — Shozab_haxor · 2026-10-26
- Google auto-converts one-off searches into AI Mode conversations, users frustrated — Status_Technician364 · 2026-10-26
- Kimi K3.1 Pro nears frontier with 65.7% Terminal-Bench score, as Moonshot deliberately trims its exploit-writing skills — maier_ak · 2026-10-26
- Polymarket opens a market on whether Moonshot ships Kimi K3.1 Pro weights by Nov 4 — Polymarket · 2026-10-26
- Orchestrating Sonnet subagents can cost more than Opus 5.5 on medium effort — Odd-Shift-640 · 2026-10-26