Dev argues models can't extrapolate user preferences even with unlimited perfect context
gabriel1 · x · 2026-09-30
Developer gabriel1 argues that personalization is not a context-size or test-time-training problem: even with unlimited perfect context, models simply can't extrapolate on a person's preferences. He finds it odd that models can't infer most preferences from how we talk, and guesses it's just low priority relative to RSI-style self-improvement.
More from AGI Musings
- Peter Diamandis talks art vs science with Neal Stephenson and Mira Lane — PeterDiamandis · 2026-09-30
- Privacy isn't dead, but much of what we call privacy today will be replaced, researchers argue — NathanpmYoung · 2026-09-30
- 80,000 Hours Founder: Automating All Cognitive Labor Dwarf the Moon Landing — ben_j_todd · 2026-09-30
- AI Safety Researcher Debunks 'Small Circle' Claims With a Base-Rate Argument — BlancheMinerva · 2026-09-30
- Plinz: capability denialists don't cite fixable limits — they claim the whole approach is doomed — burny_tech · 2026-09-30
- Anthropic's Amanda Askell: caught between safety skeptics and capability deniers — burny_tech · 2026-09-30