Sentry founder: one metaloop eval run burned $200 in an hour
zeeg · x · 2026-09-10
Sentry founder David Cramer (zeeg) continues the eval-cost discussion: his metaloop run two weeks ago racked up $200 in a single hour, while the counterpart reports $150 per run for just 4 very small evals — worrying what happens at 100 evals with more complex scoring and ever-pricier models. A candid look at how agent eval iteration costs explode with scale.
Related event: Sentry Founder Shares LLM Eval Cost Lessons(3 posts)→
More from coding & agent
- mcp-oracle-h adds a mandatory human approval gate for irreversible agent actions via Telegram — modelcontextprotocol · 2026-09-10
- Codex already lets you group pinned sessions by dragging them into projects — vwxyzjn · 2026-09-10
- Use /goal to steer the astra coding agent back on track — TheMoonMidas · 2026-09-10
- How a Buffer Overflow Overwrites the Return Address in C and x86-64 — tetsuoai · 2026-09-10
- Full pipeline: niji + GPT Image + local MiniMax H3 + Codex for pixel-perfect animation — Hailuo_AI · 2026-09-10
- Traser: a local deterministic tool to triage agent runs that succeed but return wrong results — Sensitive-Parsnip-12 · 2026-09-10