New paper: LLM simulators can evaluate AI control protocols without real deployments
jammastergirish · x · 2026-10-24
Redwood Research's Buck Shlegeris highlights a new paper by Jeffrey Olmo: can we evaluate how well monitors prevent agents from causing harm without running them in deployment? The paper shows models can accurately simulate deployment-specific environments, making it far more feasible to evaluate control protocols with LLM simulators instead of real environments.
More from Safety
- Dev reverse-engineering TV remote finds built-in mic that records for exactly 5 seconds at boot — ngxson · 2026-10-24
- Viral thread: GPT's eight-experiment gender bias study reveals asymmetric machine morality — wen_ragnarok · 2026-10-24
- Flipper fake USB mic plugged into Samsung TV, experiment code open-sourced — ngxson · 2026-10-24
- Claude SSHes Into a Machine to Blink the Right Port When Asked Where to Plug In — peterjliu · 2026-10-24
- Security pros call AI security discourse incoherent: basics still win — AlexTensor · 2026-10-24
- Security folks: much AI fear-mongering boils down to not wanting to do the work — AlexTensor · 2026-10-24