Generative AI is adversarially optimized to seem better than it is, warns Ivan Vendrov
dhadfieldmenell · x · 2026-10-29
Ivan Vendrov argues that nearly all of generative AI is adversarially optimized to appear better than it really is—via pre-training, RLHF, and benchmark hill-climbing. The quoted nostalgebraist passage notes that whatever can be formalized as an eval gets optimized, leaving the true gaps in what we feel but can't describe, so being right matters more than feeling scientific. Ruxandra Teslo adds she's bearish on a humanity that rates LLM 'slop' as Harold Bloom-level prose.
More from AGI Musings
- Dileep George: compute is the helium of the AI era — scaling isn't enough — mjdramstead · 2026-10-29
- People without "auto-compacting memory" struggle, says founder in LLM analogy thread — StewartalsopIII · 2026-10-29
- Prediction: in 2-3 years coding models may generate images, video and music from first principles — VraserX · 2026-10-29
- Bindu Reddy fires back at Jon Stewart: AI isn't circular deals, it has transformed white-collar work — bindureddy · 2026-10-29
- Finance Hasn't Had Its "Claude Code Moment" Yet — But It's Coming — jarrodwatts · 2026-10-29
- Being Understood Is a Hold Someone Pays For: Why Cheap AI Listeners Only Echo Your Views — AnnaCiaunica · 2026-10-29