ASI as a Black Box: we can deduce almost nothing about what superintelligence could do
zetalyrae · x · 2026-11-16
zetalyrae's essay argues ASI remains a gigantic unknown: the limits of our predictions come not from ignorance about ASI but from ignorance about the tasks themselves. Human geniuses are a lower bound, not an upper bound; only in domains with deductive theories (no FTL travel) can we rule things out. Whether "superpersuasion" is possible or how fast ASI could accelerate science — the "intelligence elasticity" of discovery — are empirical questions we can't currently answer. "Things an ASI can do" is a subset of a larger unknown set.
More from AGI Musings
- Pedro Domingos: only the next AI generation can understand the spaghetti code today's AI is producing — pmddomingos · 2026-11-16
- Gary Marcus doubles down: AI tools are symbolic, making the whole system neurosymbolic — GaryMarcus · 2026-11-16
- Paper: markets may defuse zero-sum status contests in a post-scarcity society — paulnovosad · 2026-11-16
- Paper: Markets Fragment Status Wars While Governments Prop Up Hierarchies — paulnovosad · 2026-11-16
- RSI Has Become a Vacuous Buzzword — Talk Capability Index Instead — maksym_andr · 2026-11-16
- Why models stumble on hard tasks: what they lack can't be described, so no RL env exists — sebpaquet · 2026-11-16