Sentry CTO: most of my CI assertions are deterministic, LLM rubrics kept minimal
zeeg · x · 2026-09-10
Sentry co-founder/CTO David Cramer shares how he uses LLMs in CI: most assertions are deterministic, rubrics are kept very minimal, and only a few non-deterministic asserts run where an agent is actually needed. A pragmatic take on agent eval design: shrink LLM-judged checks to the bare minimum and keep the rest deterministic.
Related event: Sentry Founder Shares LLM Eval Cost Lessons(3 posts)→
More from coding & agent
- Hermes Agent routing flaw silently hijacks models to metered OpenRouter, costing users money — WolframRvnwlf · 2026-09-10
- mcp-oracle-h adds a mandatory human approval gate for irreversible agent actions via Telegram — modelcontextprotocol · 2026-09-10
- Codex already lets you group pinned sessions by dragging them into projects — vwxyzjn · 2026-09-10
- Use /goal to steer the astra coding agent back on track — TheMoonMidas · 2026-09-10
- How a Buffer Overflow Overwrites the Return Address in C and x86-64 — tetsuoai · 2026-09-10
- Full pipeline: niji + GPT Image + local MiniMax H3 + Codex for pixel-perfect animation — Hailuo_AI · 2026-09-10