Dev pits OpenAI Decisions API against Jev, Clef and Polymarket — models just copy the market
Physical_Pepper6294 · reddit · 2026-10-08
- After OpenAI released its beta Decisions API, a developer built an open-source experiment comparing three "decision models" — OpenAI Decisions, TypeSafe's Jev, and Cloudflare's Clef — on Polymarket/Kalshi questions: each reads the same filtered news and outputs a probability, compared against real-money market prices, with per-article replays of how each model updates.
- Key finding: models instantly copy the market once it's in context. On a Fed question, no market price → P(cut) 0.63; "market prices 30%" → 0.30; 71% → 0.71; 95% → 0.88. So "AI vs market" is fake unless every odds-quoting source is stripped before models read anything.
- Distinct styles: Decisions is the most decisive and willing to disagree with the market; Jev barely moves; Clef reacts hardest — an SEO spam page swung it from 8% to 63% before it self-corrected.
- Stack: Next.js 16 + React 19, OpenAI Decisions API (gpt-6-luna), Cloudflare Workers AI for Clef (open weights, runs locally via Ollama), Vercel AI Gateway, Valyu search API, Polymarket/Kalshi public APIs. 27 decisions per question in 2 seconds; board refreshes every 15 minutes. Author cautions tiny sample and no resolved markets yet.
Related event: OpenAI's Decisions API Falls Short of TypeSafe's Jev in Benchmarks(4 posts)→
More from coding & agent
- Cloud agent runtime plus Tailscale: the "local hands" setup devs prefer — blelbach · 2026-10-08
- Rocket League-style game built with PlayCanvas and Claude runs inside Snapchat as a download-free Lens — willeastcott · 2026-10-08
- PyTorch's ezyang: Claude knows variational types and implements them for you — ezyang · 2026-10-08
- repo2graph turns codebases into graph context for coding agents — and shows where grep still wins — JeremyCMorgan · 2026-10-08
- New multi-agent swarm writeup: smarter models benefit more from collaborative work — repligate · 2026-10-08
- VibeBuddy is a $59 desk gadget that watches your coding agents and speaks up when they need you — juntao · 2026-10-08