How Rippling went AI-native in 6 months with Deep Agents and a 3-layer eval pipeline
LangChain · x · 2026-08-27
LangChain's case study: workforce platform Rippling shipped AI across every product line in 6 months, now serving millions of users.
Architecture: A multi-agent design built on Deep Agents — a supervisor coordinates specialized read, RAG, and action agents — to reason across thousands of tables and hundreds of thousands of fields spanning HR, IT, payroll, and finance, where concepts like "balance" are ambiguous across domains.
Eval pipeline (LangSmith):
- Offline evals: pre-recorded mocks + fixtures run locally on every commit with no external dependencies;
- Post-merge integration evals: 300-400 queries against a full sandbox to validate system health;
- Deploy-blocking evals: 10 critical scenarios against real systems gating every deployment;
- Continuous automated analysis with traces and a semi-automated self-healing loop to catch regressions.
Related event: Ripping Went All-In on AI in Six Months with Deep Agents and Layered Evals(2 posts)→
More from coding & agent
- AI software factories hinge on the Factory Loop, not faster code generation — Pavan_Belagatti · 2026-10-16
- 'Dropping a 3,000-line PR twenty minutes after ticket assignment' — AI coding meme goes viral — facontidavide · 2026-10-16
- Self Labs launches Agent Vault to let AI agents log in and pay with credit cards — EXM7777 · 2026-10-16
- Plane Agents: triggers, not prompts, are what turn chatbots into workflow agents — JosephJacks_ · 2026-10-16
- Claude eng lead Felix Rieseberg publishes tutorial on being more ambitious with AI — danshipper · 2026-10-16
- Qwen Code Desktop v0.25.2 ships managed-agent fixes and MCP config repair — github-actions[bot] · 2026-10-16