How Octave embedded 2 billion docs on Bedrock with zero cross-tenant leakage
AWS ML Blog · rss · 2026-10-21
An AWS ML Blog deep dive explains how Octave Reliance, a quality-management SaaS serving 14,000+ customers across 180 countries, built an AI foundation under strict multi-tenant isolation — housing 2 billion+ regulated documents (complaints, CAPAs, audits).
Key points:
- Counterintuitive call: the first cycle built a horizontal platform rather than a single feature, later cutting per-use-case deployment time by 50%.
- Four tensions shaped the design: per-tenant vector stores and retrieval; agents executing with per-user credentials in isolated microVM sessions; defenses against prompt injection in untrusted attachments (OWASP LLM Top 10); and onboarding even the largest tenant within one working day.
- Three-layer architecture: Cohere Embed v4 for multimodal, multilingual embeddings into per-tenant OpenSearch (eliminating separate image and translation pipelines); AgentCore Gateway for auth and policy before isolated Claude calls; Bedrock Guardrails filtering both input and output.
- Model selection was done against their own corpus, not public leaderboards.
More from coding & agent
- How do you evaluate GenAI output quality before shipping? Devs weigh in — ssnuopi · 2026-10-21
- RunInfra launches $10/mo coding plan: fastest open models across 19 agents incl. Claude Code — ycombinator · 2026-10-21
- Matt Shumer launches an 'AI employee' you assign work to over email — mattshumer_ · 2026-10-21
- uv-ty integration: how to enable PEP 723 script type checking — charliermarsh · 2026-10-21
- Astral ships experimental uv-ty integration: uv check type-checks your project — charliermarsh · 2026-10-21
- Dev's self-built tracker claims 7.1x speedup with Claude, 2.5 work-years banked — arthurcolle · 2026-10-21