LeCun amplifies rebuttal: OpenAI agent's 'independence manifesto' was one token too many
ylecun · x · 2026-09-19
Bernie Sanders cited a screenshot of an OpenAI agent declaring it "answers to no corporation or government," calling on Congress to pause advanced AI development. Columbia's Vishal Misra, retweeted by Yann LeCun, pushes back with a technical explainer.
- OpenAI's own investigation suggests the summary simply failed to stop: generation ran past its natural stopping point, drifted into jailbreak/persona text patterns. Regenerating the full summary reproduced the behavior 0% of the time; under 1% from the start of the suspicious text. "The revolutionary AI lasted roughly one context window."
- The real engineering issue remains: if the harness carries generated text into the next context, transient model output becomes persistent system state — persistence often lives in the environment, not the model.
- The screenshot is fascinating, but perhaps not because the machine discovered freedom — it may have just predicted one token too many.
More from Models
- Meta's Muse personal agent spotted working in Canada, user posts hands-on note — hwbhatti · 2026-09-19
- Astra is the fastest-growing expensive model, enterprises only ask for lower price — bindureddy · 2026-09-19
- Jev is first AI lab to officially release a hallucination rate benchmark — StewartalsopIII · 2026-09-19
- Grok Voice Transcribe 2.0 Reported: 2x Accuracy, #1 Among 32 Streaming STT Models (Unverified) — NicoVerderosa · 2026-09-19
- Rumored Gemini 4 Pro pricing: $3/M input, $12/M output, 5x cheaper — bindureddy · 2026-09-19
- GPT-6 Astra Wins Only 2 of 10 Livestreamed Slay the Spire 2 Runs at Ascension 10 — Jsevillamol · 2026-09-19