Open-Source All-in-One Enterprise AI Stack
alichherawalla · reddit · 2026-07-13
The author refers to their open-source solution as an "AI version of AWS": using a single docker compose to assemble the open-source components needed for enterprise AI deployment into a runnable whole, aiming to make organizational AI compliant, auditable, and observable.
Core components include:
- LiteLLM as the model gateway, unifying the OpenAI-compatible interface to connect local or cloud models
- LLM Guard + Presidio for PII desensitization, prompt injection/toxicity/key detection, and other guardrails
- OpenBao for key management, Langfuse for observability and tracing, OpenSearch for auditing and SIEM
- Marquez for data lineage, Temporal for persistent agent execution
- Qdrant for vector retrieval, Airbyte + dbt + ClickHouse + Great Expectations for data flow, warehousing, and quality, Kestra for orchestration, Ragas + Evidently for evaluation and drift monitoring
The author believes the real breakthrough isn't these components themselves, but tying them together into a single suite overlaid with an enterprise interface like lovable / bolt.new / replit.dev: by defining pipelines and RBAC upfront, employees can use natural language to build constrained workflows and applications; reviews, human-in-the-loop, and autonomous agents are all confined within established rules.
Related event: Open-Source Enterprise AI Stack Aims to Be “AWS for AI”(2 posts)→
More from coding & agent
- Multiagent v2 playbook calls for 64 agents, diverse proof routes and adversarial checks — danshipper · 2026-07-21
- Hermes Agent adds built-in Word, Excel, PDF and PowerPoint support — Teknium · 2026-07-21
- Super Proxy open-sources a self-hosted multi-provider LLM gateway with fallback and cost caps — Delicious-Flan88 · 2026-07-21
- Open-source MCP server connects Screener.in to live financial data for LLM research workflows — ashutosh_811 · 2026-07-21
- Marker 2 launches with PDF-to-Markdown support and claims up to 27 pages/sec — VikParuchuri · 2026-07-21
- Belgie lets Python developers build React MCP apps without installing Node.js — TheRealMrMatt · 2026-07-21