Baseten webinar: evaluate open models for coding agents by benchmarking on your own merged PRs
baseten · x · 2026-10-27
Baseten is hosting a webinar (Oct 28) with Alex Ker and Molly Elfers on how engineering teams can evaluate whether open models are good enough for production coding workloads:
- Clarify rollout objectives (cost, quality, ease of adoption) and cost-reduction levers beyond per-token pricing
- Which open models to shortlist today and how to map them from the closed models your engineers already use
- How to scope a pilot that keeps existing harnesses like Claude Code and Codex
- Build a benchmark from your own merged PRs and measure cost per successful task
- Performance and reliability metrics to check before rollout
Attendees get a detailed template to run their own evaluation. Context: coding agents are one of the fastest-growing line items for engineering orgs, and open models now perform close to frontier closed models at a fraction of the cost.
More from coding & agent
- Cursor launches Agent SDK: agents as code with Markdown prompts and TypeScript tools — soleio · 2026-10-27
- Vibecheck, the 'usable formal methods' hackathon, sells out two floors in SF — spikedoanz · 2026-10-27
- How do you verify AI agent outputs beyond unit tests and LLM-as-judge? — Practical_Green5775 · 2026-10-27
- AI agents may soon become the only way to run code, not just write it — IgorCarron · 2026-10-27
- Hermes Agent used as browser QA tester finds 8 issues in a bug-seeded site — techlatest_net · 2026-10-27
- Open-source FableCut lets AI agents edit a real video timeline via remote MCP — FastPresence9799 · 2026-10-27