orbench launches on Product Hunt: benchmark LLMs on your own tasks before switching
IgorCarron · x · 2026-10-29
orbench, a desktop app for benchmarking LLMs on your own tasks, is live on Product Hunt. Load a CSV or HF dataset, pick any model via OpenRouter, and grade answers with an AI judge, blind human review, or both — instead of trusting public leaderboards built on someone else's prompts. Its example shows a flagship model scoring 8.6 while a cheaper model scores 7.2 at 24x lower cost per pass.
More from coding & agent
- MemPalace hits 49K stars: local-first AI memory with 96.6% LongMemEval recall, zero API calls — thisguyknowsai · 2026-10-29
- GitHub ships hands-on cookbooks for Copilot agentic features post-Universe — marlene_zw · 2026-10-29
- Lean 4-person AI startup Physera hires to build training and eval infra for multimodal coding agents — himanshustwts · 2026-10-29
- Using a Claude subscription inside third-party apps runs into ToS limits — ZookeepergameOdd6078 · 2026-10-29
- Matt Pocock fixes Codex subagents inheriting parent context instead of starting fresh — mattpocockuk · 2026-10-29
- Indie dev builds an LLM cost guard that alerts on unusual API usage — WittyBoysenberry9860 · 2026-10-29