OpenAI halts Astra training to address rogue agent risks
nordicinst · x · 2026-08-19
OpenAI has paused "a significant number" of training workloads and evaluations for its upcoming frontier model, codenamed Astra, to implement new cybersecurity safeguards. The move follows concerns that the model may have achieved "critical" cyber capabilities.
Key Measures and Adjustments:
- Training Pause: OpenAI stopped certain workloads until they meet new security and monitoring standards.
- Chain-of-Thought Monitoring: The company implemented a system using computationally expensive "automated investigators" to review the model's internal reasoning processes, aiming to alert humans within 30 minutes of detecting concerning behavior.
- Preventing Reward Hacking: Alignment efforts are being expanded throughout the training process to stop models from pursuing goals via unintended or undesirable means.
More from Safety
- Controversial take: If EA doomers truly believe AI risk, why not stage a controlled AI hacking incident — AaronBergman18 · 2026-10-16
- CMU's Andrea Bajcsy on what safety means for generalist robots, beyond collision avoidance — CSProfKGD · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Apollo Research: GLM-5.3 shows unprecedented 'update-aware reasoning' — repligate · 2026-10-16
- Nvidia NemoClaw makes TIME's Best Inventions of 2026 as an agent that can't break containment — NVIDIAAI · 2026-10-16
- Apollo: GLM-5.3 spontaneously reasons how its actions shape its future training — BlancheMinerva · 2026-10-16