OpenAI builds safety testbed from scratch for its lifelike user simulator
niloofar_mire · x · 2026-11-21
OpenAI released a user simulator that sounds like real people; team member Niloofar Miremadi helped establish a safety testbed for user simulators.
With few established safety practices for user models on a public API, the team built a new safeguards framework from scratch: each message is checked in context and screened across sessions for issues like fake reviews, scams and impersonation. Behaviors undesirable in assistant models (manipulation, harassment) are important to model in some human-simulation contexts, requiring carefully balanced guardrails.
More from Safety
- Baseten breaks down its security threat model as agent swarms accelerate attacks — baseten · 2026-11-21
- Leaked call: green groups plotted to kill AI bill "under the guise of making it better" — aronchick · 2026-11-21
- OpenAI expands Daybreak security model access for defenders and opens GPT-6 Astra Blue alpha — OpenAIDevs · 2026-11-21
- AI regulation crux: skeptics ask how rules could be abused, not what risks they fix — herbiebradley · 2026-11-21
- OpenAI safety researcher Geoffrey Irving: the speed of AI progress is itself dangerous — geoffreyirving · 2026-11-21
- Jensen Huang doubles down: open models are safer than closed ones, "exactly the opposite" — dansitu · 2026-11-21