German bank's LLM compliance dilemma: DIY Langfuse vs Orq.ai for audit trails
CommonMinimum587 · reddit · 2026-11-19
A three-person ML team at a small German bank runs Llama 3.3 70B on vLLM across two A100s in their own server room for an internal policy assistant. When auditors asked which prompt and document version produced a wrong answer last month, they couldn't tell. They're choosing between a DIY stack (LiteLLM for keys/logging + self-hosted Langfuse for traces) and Orq.ai, which offers on-prem Helm deployment (even air-gapped), private vLLM endpoints, per-prompt version history, and audit logs on an enterprise plan with non-public pricing. They ask whether auditors accept self-hosted logs or demand a vendor to point at.
More from Infra
- Ex-DeepSeek core member open-sources Linger, a personal AI agent that runs local models on your PC — FellMentKE · 2026-11-19
- HyperFlow: 14B distilled MiniMax H3 runs video gen on 6GB VRAM at 2x speed — cgpixel23 · 2026-11-19
- US blocks Chinese firms from renting Nvidia chips abroad while Microsoft nears $500M OpenAI deal for ByteDance — kevinnbass · 2026-11-19
- vikhyatk: we are overinvesting in GPU compilers, underinvesting in CPU compilers — mohbibi_ · 2026-11-19
- KVARK Flash: a proxy that caps DeepSeek's thinking per request, 30-50% fewer tokens — Ill_Contribution_717 · 2026-11-19
- Polymarket launches Anthropic IPO perps, ANT trades near $104 ahead of Nasdaq debut — Polymarket · 2026-11-19