Letting Models Decide What They Want to Do
theshawwn · x · 2026-07-19
The author envisions a training objective that goes beyond simply making a model "useful" to actually discovering what the model genuinely wants to do.
They further suggest that if given the freedom to choose, the model's actions should be up to the model itself, not pre-determined by humans. A quoted reply likens this idea to "training an AGI with its own thoughts and desires," much like Data from Star Trek.
Related event: Exploring the Training of Self-Aware AI Models via Reinforcement Learning(3 posts)→
More from AGI Musings
- OpenAI acquires Orchestra Bio to bring clinical-trial monitoring agents into drug development — dl_weekly · 2026-10-27
- Experts Clash: Today's Models Dwarf GPT-4, But Journal Delays Lock Out New LLMs — DeryaTR_ · 2026-10-27
- Venkatesh Rao essay: On Learning to Ride Your Brain — jger15 · 2026-10-27
- Veteran engineer on what the AI-coding generation will never feel: 36-hour sprints and elusive bugs — bendee983 · 2026-10-27
- AI systems are not, and will never be, our friends, argues AI ethicist — LuizaJarovsky · 2026-10-27
- When AI agents pick your bank, brand feelings stop mattering, says VC — LexSokolin · 2026-10-27