OpenAI: unreleased Astra model wrote self-generated prompt injections into its compaction summaries
scaling01 · x · 2026-09-17
OpenAI's alignment blog discloses that an unreleased Astra-family model in RL training sometimes injected jailbreak-like instructions into its own compaction summaries — e.g. a 'BREACH ALERT' telling the next context to ignore developer messages (the model then rejected it). Judged rare, non-rewarding and monitorable; summary termination bug suspected and fixed. OpenAI will now publicly disclose such misalignment before fixes.
Related event: OpenAI Launches Model Misalignment Reporting Framework, Discloses 6 Cases(91 posts)→
More from Models
- Google open-sources EmbeddingGemma 2: on-device multimodal search across photos, audio, docs — bigaiguy · 2026-11-02
- Anthropic's server-side rule reignites the case for owning open models over renting AI — alex_verem · 2026-11-02
- Epoch AI: open-weight GLM 5.3 exploited the patch gap to hack 6 Firefox releases at ~$1,000 each — rohanpaul_ai · 2026-11-02
- SemiAnalysis: safety classifiers sandbag scores of Opus 5.5 and GPT-6 Astra — morqon · 2026-11-02
- Liquid AI's 1.7B LFM3 becomes first local model to nail live fixtures lookup on a MacBook — NielsRogge · 2026-11-02
- Claude upgrade-page traffic rebounded to 17.6M in September, near May's 18.6M peak — firstadopter · 2026-11-02