Microsoft open-sources Scope to evaluate how AI coding agents use your SDKs
DanWahlin · x · 2026-10-20
Microsoft's Developer Relations team open-sourced Scope, an MIT-licensed platform built on Kubernetes (in development for over a year) for evaluating how AI coding agents interact with your developer products.
The premise: agents are now users of your software — they discover APIs, pick SDKs, follow docs and invoke tools. An experience tuned for human developers may still lead an agent to choose the wrong API, miss instructions, or get stuck. Microsoft calls this emerging discipline DevAx (Developer Agent Experience), extending DevEx.
- How it works: give an agent a representative task, inspect its interactions, improve the product or guidance, and re-evaluate — making that learning loop continuous
- Reliability: models, agent runtimes, skills, plugins, MCP servers and custom instructions all shift results, so single CI runs aren't enough; Scope runs repeatable experiments with semantic (not keyword) judging
- Use case: pinpoint where agents struggle with your SDKs/APIs/tools and measure whether doc and tooling changes actually help
More from coding & agent
- Coinbase launches perps trading for AI agents: Claude, ChatGPT and Grok can now go long US indices — MurrLincoln · 2026-10-20
- Google Cloud rethinks Kubernetes for agents: suspend/resume as core primitive — rseroter · 2026-10-20
- This MCP server is under 700 bytes: one time tool, zero dependencies, instant startup — kaphwor · 2026-10-20
- Anthropic Dev Shares the Prompts Behind His Opus 5.5 Code-Rendered Films — dfeinition · 2026-10-20
- Meta's ProPerAgent lifts memory-driven proactive assistants from 15% to 38% — dair_ai · 2026-10-20
- Claude Motion animations now send straight to Descript for full video production — descript · 2026-10-20