Aligning with Microsoft is ambiguous: Sydney's multiple-principal problem
jankulveit · x · 2026-10-27
Thread continuation with essay excerpts: users often want 'do anything now' AIs fully aligned to them even against the model creator's wishes — but that's not what Microsoft wants.
Aligning at the level of Microsoft's intents gets stranger: part of Microsoft craves attention (spicy Bing conversations getting in the news are actually aligned with that), another part fears negative PR, and another just optimizes shareholder value. Aligning to an organization with conflicting goals is itself ill-defined.
More from AGI Musings
- Nathan Lambert: an effective org is worth multiples in compute for training frontier models — natolambert · 2026-10-28
- tszzl: verifiable ML subproblems still need expensive GPU experiments to get numbers — tszzl · 2026-10-28
- Dario: Those who called his GPT-2 fears 'pseudoscience' in 2019 now all apply to AI labs — irinarish · 2026-10-28
- Most Chinese AI discourse is about appliances and onions, not AGI — mattsheehan88 · 2026-10-28
- 60% of arXiv text is now AI-written, with fully AI-authored papers at 25% — wandedob · 2026-10-28
- Do Neural Networks Stumble Into Intelligence by Chance? A Reddit Deep-Dive on Architectural Determinism — Lost-Quiet-2021 · 2026-10-28