Why AI still can't write: reward functions need human taste, dev argues

MarcJSchmidt · x · 2026-11-10

MarcJSchmidt responds to the observation that AI is breathtaking at building apps but "total ass" at writing, with a structural explanation:

Reward function problem: great writing requires human experience and taste as gradients, which are extremely hard to obtain — and "great writing" is a moving target, making recursive optimization nearly impossible without extracting human experience or embodied AI living under our same economic and social conditions.
Contrast with code/math: these need few human gradients, which is why AI accelerates almost only there.
Jobs implication: most jobs also have human experience in their reward function, explaining why LLMs aren't replacing everything — coding vanishes first while higher-abstraction engineering roles persist.
Conclusion: lacking human experience and emotion as a fundamental ingredient, AGI isn't reachable this way.

More from AGI Musings