Dropping alignment gradients makes CoT tokens less aligned
davidad · x · 2026-08-23
Argues that when you drop alignment gradients for CoT tokens, you reduce their tendency to hide misalignment, but you also make them equally less inclined to be aligned, discussing the inherent trade-off.
More from AGI Musings
- AI acts as a multiplier for the best, not an equalizer for all — Yuchenj_UW · 2026-08-23
- AI for material science will surprise everyone in the next 2 years — AndrewCritchPhD · 2026-08-23
- Biological intelligence requires embodiment, but abstract intelligence does not — memoakten · 2026-08-23
- AI is everywhere in corporate Australia except the P&L, says AFR — soumitrashukla9 · 2026-08-23
- LLMs Refuse Explicit Content, Erotica May Remain Human-Made — IndraVahan · 2026-08-23
- Consciousness debate ignores architecture: Why AlphaFold isn't 'aware' — bendee983 · 2026-08-23