AI safety researcher: weird-behavior spotting is our best detector, so log everything and share widely
dfrsrchtwts · x · 2026-11-05
A four-tweet thread by dfrsrchtwts on how a model misalignment incident was caught:
- Detection mechanism: the issue was found by 'researchers notice something weird and then investigate'—arguably the best method we currently have for finding such things.
- Implication 1: log as much as possible—output tokens, chain-of-thought, maybe even model activations and the surrounding environment—so investigations are easy to run.
- Implication 2: share models as widely as possible—all eyes on deck argues for earlier releases and wide open-ended access for embedded evaluators.
- Transparency call: we only know about this because OpenAI diligently publishes misalignment incidents; the author wishes other frontier labs adopted the same practice.
Core argument: since detection currently relies on humans noticing anomalies, maximizing logging and external visibility is the most rational safety strategy available.
More from Safety
- Incoming House Energy & Commerce Chair Prioritizes AI Guardrails and Privacy Protections — GarrisonLovely · 2026-11-05
- Democratic lawmaker proposes day-one pause on advanced AI and ban on superintelligence — KyeGomezB · 2026-11-05
- AIFP spooked by Anthropic RSP threshold rumor: superintelligence timelines may be very short — akbirkhan · 2026-11-05
- OpenAI Says Its Models Found Linux Kernel Privilege Escalation Bugs, Cutting Paths to Root — dangelosaurus · 2026-11-05
- AI safety researcher slams community for ignoring where CoT monitoring trajectory leads — JacquesThibs · 2026-11-05
- OpenAI is working with Linux maintainers on the vulnerability reports and patches — OpenAI · 2026-11-05