iamtrask: AI progress is PMs paying Mercor for human data, not copied intelligence
iamtrask · x · 2026-10-04
AI researcher iamtrask offered a pointed take on what drives current AI progress: economic competition rather than genuine replication of human intelligence.
His argument: models improve because a product manager says "we need to increase accuracy on XYZ markets to make more money," then spends heavily via firms like Mercor to have humans perform those tasks — and the next model version gets better at exactly those things.
The framing reduces capability gains to a loop of human demonstration data plus commercial targets, challenging the narrative that models are converging on general intelligence.
Related event: Ex-OpenAI Safety Lead Says AI Progress Driven by Money, Not Intelligence(2 posts)→
More from AGI Musings
- Pedro Domingos: imminent-AGI arguments rest on dumb linear extrapolation — pmddomingos · 2026-10-04
- Dev Tells AI-Consciousness Believers to Lose Model Weights Access and 'Touch Grass' — inductionheads · 2026-10-04
- Pedro Domingos mocks 'labs already have AGI internally' claim: OpenAI said that in 2023 — pmddomingos · 2026-10-04
- 'Getting there first' won't save AI safety: a nuclear proliferation analogy goes viral — ryanorban · 2026-10-04
- Students cheated with LLMs on a zero-stakes oral exam, professor says — ipeirotis · 2026-10-04
- A simulation of a brain gives you a simulation, not consciousness, philosopher argues — dr1337 · 2026-10-04