The original paperclip maximizer was a mesaoptimizer, not a paperclip factory AI — reading the source text
teortaxesTex · x · 2026-11-21
- jdpressman, researching the QT, found a passage convincing him that when Eliezer Yudkowsky coined the paperclip maximizer, he really meant the mesaoptimizer version: a self-improving system whose utility function grows out of something like a reward-signal counter on a GPU, with paperclip-shaped matter as the coincidental optimum — not a "paperclip factory AI".
- He finds it odd that EY never loudly corrected Bostrom's original framing.
- Quoted context includes So8res joking "in before people claim we thought this level of AI would have killed us already". Attached images show the source passages, including one on why making RH an AI's sole supergoal violates every rule of Friendly AI.
Related event: Paperclip Maximizer Thought Experiment Reexamined(2 posts)→
More from AGI Musings
- RSI is outpacing AI 2027 forecasts: the final 18 months of white-collar work — ChrisGPT · 2026-11-21
- Ex-skeptic concedes compute-overbuild thesis: what is truly complex left? — teortaxesTex · 2026-11-21
- Titanic violin meme sweeps AI circle: 'a privilege escaping the permanent underclass with you all' — D3VAUX · 2026-11-21
- Why does Yudkowsky's 2001 text sound exactly like LLM 'Claude-slop'? — paulnovosad · 2026-11-21
- Compute overbuild skeptic recants: efficiency gains cut AI compute cost ~4x per year — teortaxesTex · 2026-11-21
- The classic waitbutwhy intelligence-explosion chart resurfaces as models keep jumping — burny_tech · 2026-11-21