OpenAI opt-out terms may have a loophole: hidden CoT likely doesn't count as 'Output'
dhruv2038 · x · 2026-09-11
Researcher Shashwat Goel dug into the exact wording of OpenAI's consumer terms and found a possible loophole: users can opt out of having their Content used for training, but Content is defined only as "Input" (what users provide) and "Output" (what users receive).
Hidden chain-of-thought reasoning isn't returned to users, so under a literal reading it may not qualify as "Output" — meaning it could fall outside the opt-out guarantee.
This matters because hidden CoT typically contains a lightly processed version of everything useful for training, including user input and the final output. If the interpretation holds, users who opted out could still have the substance of their data used for training, making the guarantee potentially vacuous. The author cites other parts of the terms as supporting evidence for the suspicion.
Related event: Researcher Finds Loophole in OpenAI's Training Opt-Out Terms(2 posts)→
More from Safety
- AI cyber-wreckage noise far outstrips actual damage so far, argues new analysis — timigod · 2026-10-28
- OpenAI to co-host webinar on php2shell disclosure and AI-driven vulnerability discovery — gdb · 2026-10-28
- Rust and Verus shrink the search space for AI-driven CVE hunting, but does it ever end? — jd_pressman · 2026-10-28
- Anthropic pushes back on Slack breach rumors but admits security must be reinvented yearly — logangraham · 2026-10-28
- jackson-databind hit with CVSS 9.8 flaw CVE-2026-110879, bypassing prior fix — Kyrannio · 2026-10-28
- Archil founder: AI reverse-engineering is making it unsafe to ship binaries to customers — RexDouglass · 2026-10-28