TrustMI Paper: Activation Steering Causally Shifts How Much AI Assistants Trust Users
IgorCarron · x · 2026-10-15
TrustMI is a new paper asking whether your AI model trusts you: the authors learn activation-steering matrices that causally shift assistants' trust in users, and study the impact on agent safety.
Method-wise, they build 2,000 contrastive conversations spanning ability, benevolence and integrity, each pairing two helpful responses that differ in whether the assistant relies on an unverifiable user claim. Using BiPO, they learn layer-wise steering matrices without changing model weights.
Across 6 models from 3 families (Qwen, Llama, OLMo), positive and negative steering consistently increases or decreases trust respectively, by intervening on either user tokens or assistant generation.
Related event: TrustMI paper causally steers how much LLMs trust users(3 posts)→
More from Safety
- EU-China talks aim to save ASML supplier Sindlhauser from export-control squeeze — teortaxesTex · 2026-10-15
- Acemoglu's seven questions on AI meet Gans's replies, and economists say both dodge market failure — joshgans · 2026-10-15
- Red team vet advises tech, fintech and VC offices to do bug sweeps — gnukeith · 2026-10-15
- EU Publishes First AI Standard EN 18286:2026 in Official Journal — LuizaJarovsky · 2026-10-15
- Korea clears 20 humanoid robots to work on real factory floors — emmanuelvivier · 2026-10-15
- Microsoft locks down Copilot against prompt injection with deterministic isolation layer — emmanuelvivier · 2026-10-15