Trask: alignment research without training data access is like finding antidotes blind
iamtrask · x · 2026-09-05
Andrew Trask argues that solving alignment requires access to an AI company's most secret asset — not weights or user logs, but training data. Trying to do alignment work without unrestricted access to training (and RL) records means feeling around in the dark: like finding an antidote without being allowed to study the poison — possible, but much harder. Consequently, he believes alignment solutions will likely only come from a tiny number of the most central, trusted researchers inside AI companies.
More from AGI Musings
- Grady Booch revives Weinberg's 1971 quip: software so buggy one woodpecker could destroy civilization — Grady_Booch · 2026-10-23
- Why Netflix keeps losing to YouTube: UGC platforms professionalize easier than pro platforms go casual — ZeroStateReflex · 2026-10-23
- AI killed the coding moat: indie devs now compete on demand and distribution — sven_ai · 2026-10-23
- DynaVelo, a neural ODE model of single-cell dynamics, published in Nature Biotechnology — anshulkundaje · 2026-10-23
- Beff Jezos: Only frontier labs out-accelerating open source can secure systems — beffjezos · 2026-10-23
- 'I don't want to live in that world': a Redditor on why AGI utopia feels meaningless — peace_venerable · 2026-10-23