Anthropic test of 52 devs: AI-assisted group scored 50% vs 67% without, error-finding worst
AlexTensor · x · 2026-10-08
A thread compiling warnings from Karpathy and Dario Amodei, anchored by an Anthropic experiment:
- Anthropic tested 52 developers learning a new coding tool: the AI group scored 50%, those without AI scored 67% — the biggest gap was in finding errors
- Dario Amodei: AI can do your job while you forget how to do it — you let it write code, fix mistakes, do the thinking, then call the result your skill
- Karpathy warns AI can feed you believable BS: ask "are you sure?" and it adds more detail, so you end up correcting others with answers you never checked
- The key question: if the chatbot disappears, how much of your "skill" disappears with it?
Takeaway: getting work done doesn't mean you learned to do it — how you use AI matters more than whether you use it.
More from AGI Musings
- Quintin Pope cites Ngo-Yudkowsky transcripts on why SGD learns dangerous search from safe domains — QuintinPope5 · 2026-10-08
- Who Owns Your AI Memory? A Case for Data Portability and Local AI — Sharon0805 · 2026-10-08
- Vernor Vinge's classic line: machines will match human intelligence, but only briefly — pwlot · 2026-10-08
- Frontier Models Decompiling Binaries Could Rescue Devices Bricked by Manufacturers — m4rkmc · 2026-10-08
- A different perspective: giving up on AI means aging will almost certainly kill us all — smith2008 · 2026-10-08
- State of AI 2026: frontier narrows to three labs, inference costs drop 13x a year — Nathan Benaich (Air Street) · 2026-10-08