Early Decisions API tests: 88.8% AUPRC on page change classification
OpenAIDevs · x · 2026-10-07
A team that beta-tested OpenAI's new Decisions API for a week found many use cases that weren't practical before: frontier intelligence at the cost of a tiny tuned classifier. It was near-perfect at filtering low-quality pages and hit 88.8% AUPRC on page change classification, with prompt tuning pushing performance further — an early real-world datapoint for the API's low-cost routing pitch.
More from coding & agent
- Dreadnode releases ScopeBench, a benchmark for agent scope adherence in offensive security — dyn___ · 2026-10-07
- Brett Adcock shows off Handoff, a computer-use agent for shopping and travel — adcock_brett · 2026-10-07
- EvalRouter adds Traces: auto-trace every agent benchmark with a few lines of code — ycombinator · 2026-10-07
- Decisions API is out; users pitch baking it into ChatGPT for real-time gameplay — flowersslop · 2026-10-07
- Feeding years of WhatsApp and Instagram data to a personal AI agent — we93 · 2026-10-07
- AgentMail launches AgentID, a 'Sign in with Google' for AI agents — ycombinator · 2026-10-07