NBER RCT: AI boosts junior patent lawyers' output by 0.6 SD but builds no lasting judgment
rohanpaul_ai · x · 2026-09-22
A pre-registered three-month RCT by David Autor et al. (NBER WP 35720) gave 133 patent lawyers at 11 US IP firms a custom AI drafting assistant, with blind expert scoring.
Work quality with AI
- Drafting quality rose 0.34 SD at 10 days (p=0.03) and 0.38 SD at 90 days (p=0.01)
- Junior lawyers gained most: +0.58 / +0.60 SD
Judgment without AI
- After three months, all participants redlined a patent application unaided
- Treated lawyers still outperformed controls by 0.32 SD (p=0.04), but the advantage was entirely among seniors (+0.45 SD)
- Juniors showed no average gain; their scores bifurcated—fewer mediocre results but more poor and more good ones
Takeaway: AI makes juniors look better without making them better at underlying judgment; foundational expertise may be a prerequisite for durable skill gains from AI-assisted practice.
Related event: RCT with 133 Patent Lawyers: AI Boosts Output, Not Judgment(2 posts)→
More from AGI Musings
- EA writer responds to critics: status is a universal currency, EAs aren't immune — AndyMasley · 2026-09-22
- OpenAI's 24-day-old internal model reportedly solved 100+ open math problems — IgorCarron · 2026-09-22
- Dev laments tech community decline: HN and Lobsters 'a shadow of what they used to be' — zetalyrae · 2026-09-22
- Nathan Young: CEA's Handling of the Bostrom Controversy Was the Wrong Call — NathanpmYoung · 2026-09-22
- "I really hate the Chinese Room": poster calls out infamous philosophy argument — AndyMasley · 2026-09-22
- MARS Magazine argues tech bosses want you terrified by AI doomsday narratives — adariostrange · 2026-09-22