Apollo researcher: training against inter-agent messages may be as harmful as penalizing CoT

amplifiedamp · x · 2026-11-17

Apollo Research's Ezra Newman published a short form arguing that training against inter-agent messages or model notes is likely about as harmful as training against chain-of-thought.

Takeaway: RL penalties on agent text traces may induce steganography and undermine alignment observability.

Original post →

More from AGI Musings

AGI Musings channel →