RCT of 133 patent lawyers: AI boosts junior output but fails to build underlying judgment
rohanpaul_ai · x · 2026-09-22
A three-month randomized trial with 133 patent lawyers across 11 US IP firms found AI-assisted drafting improved output for everyone, with juniors gaining the most (+0.58 and +0.60 SD at 10 and 90 days, vs +0.30 SD for seniors). But when AI was removed for a judgment-heavy redlining task, the pattern reversed: seniors who had used AI scored +0.45 SD over controls while juniors showed essentially no gain (-0.03 SD). AI makes junior professionals look better without making them better at underlying judgment.
Related event: RCT with 133 Patent Lawyers: AI Boosts Output, Not Judgment(2 posts)→
More from Research
- Frontend design framed as visual agent task with groupwise relative grading — stochasticchasm · 2026-09-22
- Team reportedly plans to open source 7,000 RL training environments — airesearch12 · 2026-09-22
- Why RL generalizes to reasoning but not literary writing, per AI researchers — phl43 · 2026-09-22
- Rethinking Policy Gradients: Score Centering Skips Importance Sampling Entirely — brandondamos · 2026-09-22
- Building one of the hardest on-policy lie datasets for Aletheia's Quest lie detection competition — hunarbatra · 2026-09-22
- Question's Gambit lifts deep research agents: GPT-5.5 hits 90.5% on BrowseComp-Plus — omarsar0 · 2026-09-22