Workshop: running agent evals in CI with pytest and GitHub Actions
Al_Grigor · x · 2026-11-09
AI Shipping Labs hosts a 2-hour online workshop "Evals in CI" on Nov 16, 14:00 CET, teaching how to wire an eval suite into CI so a PR that makes an agent worse can't merge. Steps: make eval runs reproducible and fast enough for CI, run them with pytest in GitHub Actions on every PR, set thresholds that fail the build on quality drops, and report results on the PR. For AI Shipping Labs members.
More from coding & agent
- NVIDIA open-sources Auto Ontology to fix text-to-SQL's biggest problem — Roger_M_Taylor · 2026-11-09
- Getting Omarchy running on Snapdragon Surface Pro: custom kernel plus SSH tunnel for OpenClaw — vincent_koc · 2026-11-09
- Resurfacing in Obsidian: Random Notes, Review Queues and Related Notes — dSebastien · 2026-11-09
- One prompt edit made an agent burn $133 on 21 rogue calls — and the 3 gates that fixed it — Worth_Wealth_6811 · 2026-11-09
- Solo dev adds read-only MCP connector so Claude and ChatGPT can query your food diary — Content-Two5729 · 2026-11-09
- Suspected DPRK npm supply chain attack: archetype-index-sdk ships obfuscated loader with credential harvesting — cyb3rops · 2026-11-09