Stanford EMNLP paper: API-level audits don't reflect what chatbot users actually get
StanfordAILab · x · 2026-09-16
Stanford AI Lab is promoting a new EMNLP paper led by Jenn Wang arguing that auditing a model via the API doesn't tell you what users actually experience in the chatbot. The authors use this as timely evidence that the third-party access debate should center on systems, not just models — the same model wrapped in different system configurations can behave very differently, so audits limited to the model layer miss real-world behavior.
Related event: Stanford Study: API Audit Results Don't Transfer to Chat Interfaces(4 posts)→
More from Safety
- Auditing my MCP credential broker: the worst hole was the config file — Relevant_Mousse_9117 · 2026-11-02
- Cambricon co-founder urges shared US-China AI ethics standards for leading models — pstAsiatech · 2026-11-02
- Vitalik's "post-AI cryptography": pseudonyms are dead, per-action anonymity is in — devanshmehta · 2026-11-02
- 6 prompts to see what ChatGPT quietly knows about you — and wipe it — Aiden_Tech_Ai · 2026-11-02
- Insider: critical infrastructure firm flunked security audit, still delays basic fixes — thedealdirector · 2026-11-02
- Cyber attack on Swedish smart fridges causes mass food spoilage at supermarket — Polymarket · 2026-11-02