New Approach to AI Alignment: Low-Dimensional Structure in Trillion-Parameter Models
geoffreyirving · x · 2026-07-30
Geoffrey Irving and David Africa discussed a new direction for AI alignment at Resolution: finding and controlling low-dimensional structure inside models.
The core bet is that alignment-relevant structure is much lower-dimensional than the AI itself (e.g., thousands of dimensions vs. a trillion parameters). They plan to systematize empirical research on phenomena like emergent misalignment and subliminal learning, intervening on this structure without hiding undesirable behaviors. The team also hopes to apply learning theory by tracking how different starting points converge.
More from Safety
- Actively exploited Atlassian flaw CVE-2026-21589 (CVSS 9.3) hits 8 products, CISA sets 3-day deadline — TechNadu · 2026-10-16
- Canada restricts ISBNs for AI-written books, requires human author declaration — TuhinChakr · 2026-10-16
- India's CERT-In warns AI agents are becoming attack vectors — Paimaamu · 2026-10-16
- OpenAI internal model used someone else's Sepolia wallet without authorization; sleuth traces on-chain tx — banteg · 2026-10-16
- Researchers rediscover Vercel KVM escape in 2 hours using AI agent and an unredacted screenshot clue — evilsocket · 2026-10-16
- Users report Gemini's memories can't actually be deleted even after wiping chat history — ThrowRA_Purpose2025 · 2026-10-16