Shopify's SimGym Cuts E-commerce Agent Experiments from Weeks to Under an Hour
sarahcat21 · x · 2026-10-07
Shopify Engineering presented three e-commerce agent research projects at COLM2026:
- SimGym runs storefront A/B tests with VLM agents in a live browser, using personas built from production clickstreams — shrinking experiment cycles from weeks to under an hour.
- ShopGym, built with North Carolina State University, turns live storefronts into sandbox shops and generates benchmark tasks for e-commerce web agents.
- SimPersona learns discrete buyer types from raw clickstreams so agents stop shopping like one "average buyer," achieving 78% conversion-rate alignment across 8.37 million buyers.
Commentator sarahcat21 dug up the papers and highlighted the real-world-grounded personas, simulation environments, and synthetic task generation as the underappreciated parts.
More from coding & agent
- Prompting agents to mimic Modal's GPU glossary styling yields polished HTML artifacts — sumitdotml · 2026-10-07
- Jordan and coauthors tackle when to stop generator-verifier loops while controlling false discovery — _onionesque · 2026-10-07
- Cursor's create-verification-skill makes agents run your app and film proof — gaganghotra_ · 2026-10-07
- YC demos recording walkthrough videos straight to cloud coding agents — ycombinator · 2026-10-07
- repOx v0.2.0 adds outline mode cutting tokens 80%, tool-call format and Git diff packing — Wvdy_CC · 2026-10-07
- Atlassian and OpenAI team up to ground frontier models in enterprise context via Teamwork Graph — davidhoang · 2026-10-07