Against 'should' and 'never': Daniel Faggella on epistemic horizons and AGI ethics
danfaggella · x · 2026-10-30
After sparring with Levi Turksma and Michael Levin, Daniel Faggella lays out his stance on AGI ethics: all claims about the "real" and the "should" come from bounded cognitive vantage points, and entirely new kinds of minds will shift those vantage points.
- Common AGI-era claims—humans must always stay in control, AI must respect autonomy, conscious entities' preferences must be honored—all presuppose our current cognitive vista
- His "Three Humility Horizons" framework maps epistemic, axiological, and categorical boundaries: bounded today, already expanded by science/reflection/evolution, and potentially radically expanded by BCI and AGI
- Values and beliefs shift with age, culture, and technology; science gives shared vantage points but hasn't erased disagreement
- He vows to keep denying the permanence of our axiological and epistemic horizons—"tents of the night, and nothing more"
More from AGI Musings
- Ethan Mollick: AI slop is killing discussion for anyone building a reputation online — emollick · 2026-10-30
- 'Genuinely thoughtlessly evil': platform slammed for unchecked AI-generated research critiques — owl_posting · 2026-10-30
- Ethan Mollick: AI slop replies bury real human responses on X — emollick · 2026-10-30
- No Man Is an Island: how AI dissolves the intellectual communities that sustain thinking — zetalyrae · 2026-10-30
- Longtime AI skeptic: at $20/month LLMs are already more useful than Netflix — CLXXXVI · 2026-10-30
- Evals Are the Most Important Skill in AI Today, Argues AI Researcher Omarsar — omarsar0 · 2026-10-30