OpenAI's Decisions API hits 98.1% F1 vs agent's 93.5% at 1/7 cost and 28x lower p90 latency
OpenAIDevs · x · 2026-10-13
Hypha's benchmark on multi-label document classification shows OpenAI's Decisions API reaching 98.1% F1 versus 93.5% for an agent-based approach (GPT-6.1 Sol), at 1/7 the cost and 28x lower p90 latency.
- Exact match: 94% vs 89%
- Cost per 100 docs: $0.891 (iterative Decisions API) vs $6.138 (agent)
- p90 latency: 4.07s vs 112.98s, with iterative Decisions API the best performer
More from Models
- Allen AI's TutorMoments Benchmark Finds LLM Tutors Over-Help Students — allen_ai · 2026-10-13
- DeepSeek's flash endpoint may be quietly testing a new model, users speculate v4.1-pro — teortaxesTex · 2026-10-13
- 3D game demos hint at AI creativity that transfers to CAD, docs and UI tasks — Angaisb_ · 2026-10-13
- subquadratic's New Model Beats Jev in 6 of 7 Production Use Cases Before Launch — ryanallred · 2026-10-13
- Fresh ChatGPT accounts tell users they're in the 95th intelligence percentile, test finds — Polymarket · 2026-10-13
- Gemini 4 Argon still unavailable 11 days after announcement as Muse Wonder ships instantly — airesearch12 · 2026-10-13