Putting an AI agent on a bank's data warehouse: the model was never the hard part
the_darkest_horse · reddit · 2026-09-19
A Southeast Asian retail bank hired an AI engineering team to let analysts query their warehouse (10 tables, 858 columns) in plain English. The obvious v1—schema in prompt, model writes SQL—worked in demos and wobbled in week two: wrong joins, wrong date filters, confident answers the data couldn't support.
What got it to production, ranked by impact:
- Three parallel candidate queries + a separate verifier call that reads the question, schema, and candidates, then picks one or rejects all. "I cannot answer this from the available tables" beat a wrong number every time in user testing.
- A hard gate on model changes: no swaps without an eval run ID and a named approver. A 60-question eval set with analysts, bar set at 85% correct under 9s median; the current model cleared it, two cheaper ones didn't—one would have shipped on vibes without the gate.
- Prompt caching on the schema prefix (9–11k static tokens) cut per-question cost more than any prompt tweak.
The bank then wanted self-hosting in its own VPC. Priced out: managed API $1k/month vs $3–9k/month in GPUs plus engineer-weeks for an OpenAI-compatible adapter. "Self-hosting is cheaper" did not survive a spreadsheet.
More from coding & agent
- Dev builds WoW mode for t3 chat to monitor coding agents mid-raid — Vjeux · 2026-09-20
- Claude Code reportedly snapshots git before every bash call to fake an edit view — steipete · 2026-09-20
- Open-source Nautilo lets your AI agent DM coworkers and fetch feedback for you — Dan_Jeffries1 · 2026-09-20
- Anthropic's Head of Product Drops a 28-Minute Masterclass on Agents in Production — ifioknkem · 2026-09-20
- HarnessRouter: routing agent harnesses instead of models, a fresh infra idea — daniel_mac8 · 2026-09-20
- MCP tool naming: short generic verbs vs explicit prefixes for LLM tool selection — skvark · 2026-09-20