On-prem MLOps in a hospital: monitoring self-built and vendor models
zentax2001 · reddit · 2026-08-22
A hospital IT professional shares insights on building an on-prem MLOps platform on an OpenShift cluster. The team is evaluating Red Hat OpenShift AI and ClearML, with the primary challenge being production monitoring. Due to EU MDR and AI Act regulations, they must implement comprehensive monitoring, including data/prediction drift, bias monitoring (specifically subgroup performance), custom metrics, and immutable inference logging.
The proposed architecture involves self-hosting Evidently AI to compute metrics and push them to Grafana. A significant challenge is monitoring third-party vendor models running on external infrastructure; lacking runtime access, they contractually require vendors to provide all input/output data feeds, which are then ingested into a local independent monitoring pipeline for validation.
More from Infra
- Article: Why America's Data Center Build-out is Facing Resistance — robleclerc · 2026-08-22
- Modular shares LLVM 2023 talk: Mojo systems programming language explained — clattner_llvm · 2026-08-22
- Chris Lattner: Modular is the "anti AI compiler" — tinygrad-style approaches never scaled — clattner_llvm · 2026-08-22
- Microsoft ONNX Runtime accelerates cross-platform ML inference — microsoft · 2026-08-22
- vLLM Conference next week: co-hosted events with Google, AMD, NVIDIA — vllm_project · 2026-08-22
- US Governors Slow Data Center Development Amid AI Backlash — pstAsiatech · 2026-08-22