How Comet improved a financial incident remediation agent's score from 0.34 to 0.57 with Opik

dl_weekly · x · 2026-11-17

Comet shares a hands-on guide to evaluating and optimizing an AI incident remediation agent for financial services using Opik. The demo spins up a realistic payments stack (API gateway, payments service, fraud scoring, card processor, Postgres ledger, FX service, Redis idempotency cache), deliberately breaks it in ten ways, then traces and scores how the agent investigates and resolves each incident. Using the Opik Agent Optimizer, the team raised the agent's overall score from 0.34 to 0.57 across 30 validation runs while eliminating unsafe actions like wrong-service rollbacks or disabled fraud checks.

Original post →

More from coding & agent

coding & agent channel →