Stripe: agents now run the ML experiment loop, blocking tens of millions in fraud a year

emilygsands · x · 2026-10-20

Stripe engineers detail how they scaled Shield, their transaction fraud model, improving recall by over 8%. After a recent fraud attack got past Shield, agents used model explanations to generate hypotheses, engineers picked the best ones, and agents tested them — the winner was a better stochastic weight averaging setup that now prevents tens of millions of dollars in additional fraud per year.

Key engineering lessons:

Shield previously used only a fraction of 800M training transactions, discarding most payments with no reported fraud; fuller history helps it distinguish fraud from business-specific legitimate behavior
Agents now run the whole ML experiment loop (data exploration, feature building, backfilling, training, debugging, evaluation); the team wrote down operational knowledge so agents can load and follow it
Training iteration cut from 24h to 4.8h per epoch via S3 streaming, torch.compile, lower precision, lower peak memory, and tuned GPU communication
Engineers are freed for judgment work: setting priorities, reviewing evidence, deciding what ships

Original post →

Stripe: agents now run the ML experiment loop, blocking tens of millions in fraud a year

More from coding & agent