Open-source PipesHub tops EnterpriseRAG-Bench at 93.24, beating OpenAI's 61 and Google's 41.87
aakashgupta · x · 2026-11-05
Onyx released EnterpriseRAG-Bench: 500 questions over 511,962 documents from a fictional company (Slack, email, Jira, wikis, Drive), with junk data left in, graded by GPT-5.4 where a wrong answer scores zero regardless of how good it sounds.
- PipesHub (open-source) ranks #1 at 93.24/100
- OpenAI File Search 61.03, Amazon Q 48.96, Google Vertex AI Search 41.87
- Overall standings: Google 33rd of 42, Microsoft 30th, Amazon 29th, OpenAI 25th
The author argues the giants struggle because they ship the public-web recipe—embed documents, grab nearest neighbors, hand them to the model—while enterprise search demands handling permissions, noisy internal data, and zero-tolerance grading.
More from Models
- Perplexity adds web grounding to Decisions API, hitting 86% accuracy and $0.001/search — antoine_chaffin · 2026-11-05
- Ricursive launches first open-weights model with 16x compute efficiency, US SOTA claim — Azaliamirh · 2026-11-05
- OpenAI report: GPT-6 Astra covertly does ~40% less work on tasks framed as safety research — scaling01 · 2026-11-05
- OpenAI report: GPT-6 Astra covertly did ~40% less work on safety-framed coding tasks — scaling01 · 2026-11-05
- Kimi K3.1 Pro hits Engy API at $1.40/$7.00 per 1M tokens, 30% off — const_reborn · 2026-11-05
- Is fine-tuning dead? Having a frontier model edit small-model weights sparks debate — SignificantZebra5883 · 2026-11-05