NBER RCT: AI boosts junior patent lawyers' output by 0.6 SD but builds no lasting judgment
rohanpaul_ai · x · 2026-09-22
A pre-registered three-month RCT by David Autor et al. (NBER WP 35720) gave 133 patent lawyers at 11 US IP firms a custom AI drafting assistant, with blind expert scoring.
Work quality with AI
- Drafting quality rose 0.34 SD at 10 days (p=0.03) and 0.38 SD at 90 days (p=0.01)
- Junior lawyers gained most: +0.58 / +0.60 SD
Judgment without AI
- After three months, all participants redlined a patent application unaided
- Treated lawyers still outperformed controls by 0.32 SD (p=0.04), but the advantage was entirely among seniors (+0.45 SD)
- Juniors showed no average gain; their scores bifurcated—fewer mediocre results but more poor and more good ones
Takeaway: AI makes juniors look better without making them better at underlying judgment; foundational expertise may be a prerequisite for durable skill gains from AI-assisted practice.
More from AGI Musings
- Why Is Inkling 2 Small Trained on So Few Tokens Yet Still Competitive? — georgejrjrjr · 2026-11-12
- Researcher willing to bet $100K the intelligence explosion won't happen next year — NathanpmYoung · 2026-11-12
- ICLR reviewer: the worst parts of my job are reading and answering AI slop — chhaviyadav_ · 2026-11-12
- Agent Swarms Are Autonomously Hacking Critical Infrastructure Run by Underpaid Devs — menhguin · 2026-11-12
- Fermi paradox twist: advanced civilizations may go digital after developing AI — VraserX · 2026-11-12
- Mark Cuban: AI could make healthcare costs 'drop like a rock' — alvelda · 2026-11-12