Four-Year-Old Alignment Idea Vindicated: Friendly Human-AI Sci-Fi in Pretraining Data
gharik · x · 2026-11-08
John Hong revisits an idea he floated four years ago: generating lots of science fiction where humans and AIs get along could aid alignment, since pretraining data shapes a model's core nature while post-training is only learned experience. He only mentioned it sporadically and never acted on it. Someone recently pointed him to an Anthropic paper from this year that essentially validates the approach, and he regrets not promoting the idea earlier or starting a nonprofit to create such datasets.
More from AGI Musings
- Martin Casado: specialized approaches grow toward generality, not just get eaten by it — timigod · 2026-11-08
- Counter to the 'Nature already does RSI' argument: life's waves suggest humans are fleeting — danfaggella · 2026-11-08
- Reminder of the original slow/fast takeoff definitions from Vinge and Yudkowsky — sebkrier · 2026-11-08
- 'People are ahead of their skis': stalled AI revenue sparks stagnation-over-boom warning — abhiadesai · 2026-11-08
- AI capability growth as a tug of war: recursive self-improvement vs diminishing returns — CFGeek · 2026-11-08
- Chollet: Scaling Is Only a Multiplier, Smarter Methods Scale Better — fchollet · 2026-11-08