AI-built OpenSearch clone validated by 6,000 black-box tests: Rust at 64.5%, Go at 32.6%
Strong-Mousse-1020 · reddit · 2026-10-26
SlopSearch is an OpenSearch-compatible server largely written by AI, where correctness is defined by tests, not the model.
Setup
- 6,100 pytest black-box tests as the spec: every test first run against real OpenSearch 3.9.0, capturing status codes, error types, response shapes, even key order
- Two independent implementations (Go and Rust) share only that suite
- Loop: pick a red area, read expectations, let AI make it green, verify no regressions; one hard rule — never edit tests to pass
Status
- Rust: 64.5% (aggregations 100%, search API 98.5%, documents/bulk 95%, ingest pipelines 93%)
- Go: 32.6%
- Weakest: text analysis and Painless scripting
Lesson: vibe coding toward a vague goal goes nowhere; vibe coding against thousands of precise, verified expectations converges — tests are the steering wheel, the model is the engine. In-memory only, no auth/TLS yet; runs in Docker with a reference OpenSearch to compare against.
More from coding & agent
- The math on shipping tiny Shopify apps with Claude Code: one app, up to $8,740/month — thisdudelikesAI · 2026-10-26
- One command wires Shopify's Dev MCP into Claude Code for hallucination-free API calls — thisdudelikesAI · 2026-10-26
- Background CUA is the future: separate execution environments and handoff matter for computer-use agents — TianbaoX · 2026-10-26
- Shopify merchants did $100B+ in Q1 alone — and AI tools just made building for them cheap — thisdudelikesAI · 2026-10-26
- Shopify takes 0% cut on your first $1M as Claude Code + MCP slash dev costs — thisdudelikesAI · 2026-10-26
- Voice input makes prompts longer and richer, but models still miss nuance — Suspicious_Orchid770 · 2026-10-26