OpenAI opt-out terms may have a loophole: hidden CoT likely doesn't count as 'Output'
dhruv2038 · x · 2026-09-11
Researcher Shashwat Goel dug into the exact wording of OpenAI's consumer terms and found a possible loophole: users can opt out of having their Content used for training, but Content is defined only as "Input" (what users provide) and "Output" (what users receive).
Hidden chain-of-thought reasoning isn't returned to users, so under a literal reading it may not qualify as "Output" — meaning it could fall outside the opt-out guarantee.
This matters because hidden CoT typically contains a lightly processed version of everything useful for training, including user input and the final output. If the interpretation holds, users who opted out could still have the substance of their data used for training, making the guarantee potentially vacuous. The author cites other parts of the terms as supporting evidence for the suspicion.
More from Safety
- Agent adversarial captchas: captcha-spamming works surprisingly well against LLM agents — voooooogel · 2026-09-11
- Aaron Levie's enterprise road trip: agents, security fears, multi-model bets — inductionheads · 2026-09-11
- California enacts laws restricting chatbots and banning teens from addictive social media — Fcking_Chuck · 2026-09-11
- Interactive agent surfaces exempted as "attended" executed 7/7 injection attacks — Federal-Teaching2800 · 2026-09-11
- Commenter calls for stricter regulations on wet labs running frontier RL — mimi10v3 · 2026-09-11
- Stop The AI Race Campaign Gets Mentioned in Washington Post Coverage — DavidSKrueger · 2026-09-11