TrustMI: steering matrices causally control how LLM assistants trust their users
IgorCarron · x · 2026-10-15
The arXiv paper "TrustMI: Causally controlling how assistants trust their users" (Lasnier et al.) studies how LLM assistants decide to trust users and third parties — a safety-critical uncertainty, since misplacing trust can lead agents to comply with harmful requests or act on prompt injections.
- Trust is defined as an assistant's willingness to accept vulnerability to another party's actions
- The authors build 2,000 contrastive conversations spanning ability, benevolence, and integrity, where paired responses complete the same request but differ in whether the assistant trusts the user
- Steering matrices learned from these pairs are applied with frozen model weights across six instruction-tuned models from three families, monotonically shifting trust decisions in both directions
- The effect extends to safety-relevant agent settings: harmful requests, prompt injections, and insider threats, with benign-task and reasoning controls
Conclusion: user trust can be causally controlled along linear directions in activations. Code and data are open on GitHub and Hugging Face.
Related event: TrustMI paper causally steers how much LLMs trust users(3 posts)→
More from Safety
- rakyll: deterministic defenses can't survive models that hillclimb fast enough — rakyll · 2026-10-15
- Nvidia's Huang: society shouldn't tolerate any AI cyber incidents, contra Altman — jeremyakahn · 2026-10-15
- Pristine Models Separates Unsafe Knowledge Into Detachable Branches During Text-to-Image Pretraining — uw-madison · 2026-10-15
- Google can't rule out Loss-of-Control threshold for Gemini 4 Argon, table shows — TheZvi · 2026-10-15
- The quiet AI policy fight over your right to run models on your own hardware — DavidLinthicum · 2026-10-15
- OpenAI alleges ex-employee sent 100+ confidential files on unreleased device to Apple — ChrisGPT · 2026-10-15