Langfuse engineer: banks certify apps, not models — gating AI agents at a top-5 bank
AI Engineer · youtube · 2026-11-04
Doneyli De Jesus, who leads AI engineering solutions at Langfuse, shares how his team helped a top-five bank gate AI agent deployments. Key points: banks certify applications, not models, requiring on-prem, gated, air-gapped setups. The approach builds a golden data set and scores traces across multiple dimensions. In one case, a small model that was 97% grounded but only 60% accurate on numbers cleared the bank's 85% gate once given a calculator tool. Domain experts set thresholds and review failures, cutting evaluation from weeks to minutes.
More from coding & agent
- Routerly 0.6.0 adds an MCP server for managing a self-hosted LLM gateway — nurge86 · 2026-11-04
- Dev pushes back on 'cognitive offloading' fear: real ceiling is attention times APM — menhguin · 2026-11-04
- Claude Code web traffic grows roughly 20x in a year to 11M monthly visits — gaganghotra_ · 2026-11-04
- Dev's 4 days with Codex: tool admits 16 failures, Claude had to finish the job — v0id_user · 2026-11-04
- Steve Yegge: ~25% of adversarial agent reviews hold up, and he's keeping them — Steve_Yegge · 2026-11-04
- Running three parallel browser Spaces for code review with ego lite — alex_verem · 2026-11-04