Human taste can be gamed into optimized slop; preference is not a verifiable reward
torchcompiled · x · 2026-08-25
@torchcompiled responds to the idea that human taste can be hill-climbed, arguing that the resulting "optimized tasteslop" is an empty supernormal stimulus wearing the clothing of authentic content's natural flow and entropy.
However, the author disagrees that preference is verifiable, listing why:
- Continuous and multi-axis: some options aren't nicely comparable; someone may like both for different reasons
- Relativity fails: a>b and b>c doesn't always imply a>c
- Changes with discovery: new content reshapes how we see old content
- Socially manipulated and conditioned
- Nonstationary long-term: boredom, desensitization, novelty seeking shift taste
- Nonstationary short-term: preference varies with current state and mood
Related event: Debate: Human Taste Can Be Gamed, Not Verified(2 posts)→
More from AGI Musings
- Debate on High-Value Models: Is Intelligence Too Cheap to Meter? — haider1 · 2026-08-25
- Stanford Exploration: LLMs Do Not Possess Emotions Despite Appearances — LuizaJarovsky · 2026-08-25
- Colleges cut supplemental essays over AI-writing concerns — Polymarket · 2026-08-25
- SF Joke: Human jobs after AI takeover — davidyin44 · 2026-08-25
- Superintelligence as Millions of Specialized Agents, Not One Model — VraserX · 2026-08-25
- Harper Carroll: The Beauty of AI and the Public Misconception — HarperSCarroll · 2026-08-25