Rachel Thomas: 'Perfect' AI agents are neither realistic nor desirable, human oversight is here to stay
math_rachel · x · 2026-10-17
fast.ai co-founder Rachel Thomas responds to skepticism about her paper's argument: AI models don't lie with moral intent — they're trained to produce plausible text, and training tends to reward "fluent, agreeable, confidence-inducing" outputs.
The paper argues human oversight of AI will always be necessary because "'perfect' agents are neither realistic nor desirable": failures are fundamental to acting in human worlds and a direct consequence of systems flexible enough to serve different users' potentially conflicting needs.
Related event: Fast.ai's Rachel Thomas: AI "Lying" Is a Byproduct of Training, Not Intent(2 posts)→
More from AGI Musings
- Alok Sikka essay: holding multiple truths at once is the skill that will build AI's future — vsikka · 2026-10-17
- Europe's AI edge may lie in the app layer: AI for aging power grids — ingliguori · 2026-10-17
- Could we control LLM reasoning vs creativity separately, like image style transfer? — unironictechbro · 2026-10-17
- willcb: the dominant career of the coming AI industrial revolution will be "plugging it in" — willcb · 2026-10-17
- Researchers warn AI-generated fake data is flooding science: experiments fabricated end-to-end — EhudReiter · 2026-10-17
- Bostrom's alignment prerequisite: to align superintelligence, you must be trustworthy — MoonL88537 · 2026-10-17