Baseten webinar: how to evaluate open models for coding agents on your own codebase
baseten · x · 2026-10-20
Baseten is hosting a webinar on Oct 28, 11 AM PT with Alex Ker and solutions architect Molly Elfers on evaluating open models for agentic coding.
Key points:
- Open models now perform close to frontier closed models at a fraction of the cost, but public benchmarks rarely reflect how they do in your own repo
- Topics: rollout objectives (cost/quality/adoption) and cost levers beyond token price, which open models to shortlist and how to map them from closed models, scoping a pilot without changing existing harnesses like Claude Code or Codex
- Building a benchmark from your own merged PRs, measuring cost per successful task, and reliability metrics to check pre-rollout
Attendees get a full evaluation process plus a runnable template.
More from coding & agent
- Perplexity's Open-Source Sandbox Platform SPACE Joins Crusoe Cloud's Managed AI Portfolio — AravSrinivas · 2026-10-20
- SpudStack uses Grok to review agent plans with PASS/FAIL verdicts — agents can't sign off their own work — alexcovo_eth · 2026-10-20
- Cloudflare MCP Server adds step-up auth: agents request scopes on demand — irvinebroque · 2026-10-20
- Skill Harbor, a skill directory for Muse agents, opens with a 10-member ecosystem — InformationOk4877 · 2026-10-20
- Nine conditions for AI to actually show up positive in your P&L — vasuman · 2026-10-20
- Three probes track post-training SWE-bench rankings, BPB hits ρ=0.964 — hyunw_kim · 2026-10-20