Astra alignment gains predate HF incident; ExploitGym Honeypot is out-of-distribution eval

kaicathyc · x · 2026-09-05

A response to controversy around the Hugging Face incident clarifies the source of Astra's alignment improvements:

The gains came from general techniques developed long before the incident; ExploitGym Honeypot was added only afterward and is out-of-distribution for RL runs.
Broader improvements show up in deployment simulations, deception evals, and realistic computer-use tasks, though failures and substantial room for improvement remain.
The author admits the system card should have better explained where alignment improvements came from, insists the lab does not benchmark-maxx alignment evals, and acknowledges that metagaming and eval awareness remain real challenges for measuring alignment.

More from Safety