Hamel Husain: How to use decision models for LLM evals, and why the 0.5 threshold default is wrong
HamelHusain · x · 2026-10-21
Hamel Husain and Antaripa Saha published a new guide, How To Use Decision Models For Evals, covering how to pick a decision model for your task, tune its threshold (don't assume the 0.5 default is good), and interpret its scores, with interactive charts throughout. The post notes decision models have exploded recently: TypeSafe's Jev, Perplexity's Decider, Liquid's d1, Cloudflare's Clef, and more.
More from coding & agent
- UC Berkeley open-sources SkyDiscover, an AI framework that evolves better algorithms and builds full systems — CShorten30 · 2026-10-21
- ruff author on ts-rust's trick: vendor and trim forked deps for a language subset — charliermarsh · 2026-10-21
- JSON prompting with MOSS Image 5B: bbox coordinates for exact multi-room layouts — cocktailpeanut · 2026-10-21
- Rewriting an agent skill: A/B challenge runs show skills should only add what agents don't know — jmschreiber91 · 2026-10-21
- Developer builds himself a 'Rork MoTS' agent on Grok Bot, aiming for full autonomy before GTA 6 — rudrank · 2026-10-21
- OpenAI Live API tip: control voice sessions from your backend via sideband WebSocket — pbbakkum · 2026-10-21