What bank security teams demand before approving LLMs: a six-point gateway checklist

Pleasant-Whole-3220 · reddit · 2026-11-17

A practitioner building a self-hosted LLM gateway shares how banks in Turkey hit the same walls: scattered provider API keys, runaway bills, no audit trail, and no answer to 'what stops someone pasting customer data into a prompt?'. Requirements before rollout: a single internal endpoint so apps never hold provider keys; per-project/key/user/model limits with hard monthly budgets; secret scanning and PII masking before requests leave the network; all checks run before the model call so blocked requests cost nothing; tamper-evident audit logs feeding the existing SIEM; everything self-hosted rather than a SaaS proxy. Asks how others solved it: in-house, LiteLLM/Portkey, cloud gateway, and what auditors unexpectedly demanded.

Original post →

More from coding & agent

coding & agent channel →