Agents can't infer your preferences from a few questions — users must teach the model
jamesbrandecon · x · 2026-09-25
James Brandecon argues there are obvious hard limits to how well any agent can infer a principal's preferences from just a few questions, with no way around it. Users will therefore have to invest real time teaching a model — not only specific preference rankings, but how to model their preferences so it can extrapolate to new situations. The observation highlights a core tension in agent personalization: passive questioning isn't enough, and user-side teaching effort is unavoidable.
More from AGI Musings
- Nina Schick: democracies must shape AI to secure prosperity, sovereignty, freedom — NinaDSchick · 2026-09-25
- Idea Machines podcast returns with Ada Palmer on building the future and sci-fi — Ben_Reinhardt · 2026-09-25
- mitsuhiko wonders which AI industry players will embrace the Super Intelligence framing — mitsuhiko · 2026-09-25
- Bill Gates warns AI is powerful enough to cause 'a billion deaths' — Polymarket · 2026-09-25
- McAfee: Silicon Valley Has Expanded Beyond Tech Into Cars, Space and Defense — amcafee · 2026-09-25
- TOWA TEI and Nakamura Takeshi on co-creating with AI: 'AI can't digest zanmi' — Merzmensch · 2026-09-25