Stripe launches coding-agent integration evals; GPT-6-Astra tops one-time payments at 90
jeff_weinstein · x · 2026-11-20
- Stripe launched Integration evals, a benchmark measuring how well coding agents build high-quality Stripe integrations, with results to be published periodically to help developers pick the best agent setup.
- It compares model-harness pairs (Claude Code, Codex, OpenCode) across three scenarios — one-time payments, subscriptions, and Connect — on a 0–100 scale.
- Standout scores: gpt-6-astra leads one-time payments at 90; claude-fable-5-2 hits 89 on Connect; subscription scores are generally lower (gpt-6-astra tops at 82).
- Methodology quantifies the impact of Stripe's official agent plugin (Stripe-authored skills + Stripe MCP tools): scores improve consistently with the plugin, e.g. gpt-6-astra 90→96 on one-time payments, gemini-4-argon 86→95 on Connect.
- Takeaway: beyond model choice, harness and the official Stripe plugin materially affect integration quality.
More from coding & agent
- Gradium MCP beta lets coding agents turn scripts into speech in-session — mattturck · 2026-11-20
- DeepLearning.AI launches Voice for AI Agents course with Vocal Bridge — DeepLearningAI · 2026-11-20
- Swarms Marketplace nears 7,000 listings with $70M+ total volume — KyeGomezB · 2026-11-20
- RedMonk launches first Agent Activity Index as coding agents upend language rankings — AccBalanced · 2026-11-20
- Dev runs 6 parallel Neon agents; benched Claude for too many safety refusals — iskander · 2026-11-20
- AI agent builds a game with local Blender and Three.js, wins 96/96 seeded runs — kimmonismus · 2026-11-20