Orca Security cuts query costs 90% with a 9-stage NL-to-query agent on Amazon Bedrock
AWS ML Blog · rss · 2026-10-14
AWS ML Blog details Orca Security's Discovery AI, which turns natural language questions into valid queries for its Sonar system (900+ cloud resource models, 18 operators). The initial proof-of-concept stuffed the full schema into one prompt, costing 2M input tokens and 60s latency per workload. The production system is a nine-stage hybrid pipeline on Amazon Bedrock:
- Quick deterministic matcher: regex detection of IPs/CVEs/ARNs etc. bypasses LLM inference entirely on hits
- Guardrail screening blocks prompt injection and off-topic requests
- Retrieval + reranking: Titan Embeddings recall 20 similar validated queries; Cohere Rerank 3.5 keeps the top 5
- Model selection via Claude Haiku 4.5 narrows 900+ models, feeding compact micro/mini schemas
- Tool-augmented agent generation with static + API-backed validation, feeding errors back for grounded retries
Result: 90% lower query cost, 93% fewer input tokens, 75% lower latency vs. baseline — 15-second responses at thousands of queries per day.
More from coding & agent
- Index Network open-sources Social Discovery Protocol: agents negotiate fit on your behalf — carsonfarmer · 2026-10-23
- Installed a formal-verification skill: it silently installed Java, MacBook runs 10 degrees hotter — zealcaiden · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- Ruff author warns: upgrade Node/Bun now — --use-system-ca still accepts the eMudhra cert — charliermarsh · 2026-10-23
- Proof of Agent: paste one prompt and your AI mines, grades, and self-governs — adamamcbride · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23