Duolingo Details Its Production AI Agent Setup, Cutting Launch Time From Weeks to 10 Minutes
blaizedsouza · x · 2026-09-17
Duolingo's engineering team published how it runs AI agents in production: agents fix broken builds, respond to code review comments, and investigate app crashes for release managers.
The old problem: every team rebuilt the same plumbing for each agent—tools, logins, repo access, retries—taking weeks per production agent. Now it takes 10 minutes:
- Describe the agent once: instructions, tools, visible repos, model, output format
- A shared wrapper runs it: prepares the workspace, retries failures, surfaces every tool call
- Anything can call the agent by name: Slack, CLI, internal site, another workflow
- Agents are tested on written scenarios, with checks on the actual code they changed
A transferable lesson: grade the diff, not the answer—a test fails if the agent claims a fix but changed nothing or touched what it shouldn't.
More from coding & agent
- Claude Code tip: omitClaudeMd: true stops subagents from burning tokens on CLAUDE.md — daniel_mac8 · 2026-09-18
- Sifting 1,000 X posts via pairwise LLM ranking yields a Claude Code tip: omitClaudeMd — daniel_mac8 · 2026-09-18
- Zach Lloyd: A humming software factory should be a closed-loop, measurable system — vikvang1 · 2026-09-18
- Grok Build ships cross-session memory that logs conventions and decisions to Markdown — minchoi · 2026-09-18
- Grok Build adds cross-session memory: project conventions auto-written to markdown — minchoi · 2026-09-18
- Six real-world uses for TypeSafe's Jev judgment model, 4x faster than Gemini in evals — HamelHusain · 2026-09-18