ORO Bench update: agents tested in realistic shop environments with public practice world
shibshib89 · x · 2026-11-04
ORO, an agent-evaluation project backed by the Crucible incubator, shipped its Oct 18–Nov 1 changelog: ORO Bench now tests agents in shops with their own products, tools and situations using pass/fail task scoring; a new public practice world allows local testing graded the same way validators grade; validators need v3.0+ and SDK clients need oro-sdk / oro-ai/sdk 7.0.0+; episode feedback and inference cost reporting are clearer.
More from coding & agent
- Anaconda on deslopping AI prose: the structural tells in PR descriptions — anacondainc · 2026-11-04
- Agent State Machines: adding deterministic scaffolding to tame nondeterministic LLM agents — blaizedsouza · 2026-11-04
- Formally Verifying Ibex RISC-V MUL/DIV Instructions with One AI Prompt — satnam6502 · 2026-11-04
- Six compliance traps that can cost vibe-coded apps six figures before first sale — aakashgupta · 2026-11-04
- Rust inference engine beats SGLang 13.4x on 250K-context agent TTFT, open-sourcing soon — ahstanin · 2026-11-04
- Phone Use Arena launches: 17 phone-use agents ranked, task costs range $0.008-$1.23 — ycombinator · 2026-11-04