Drex's calibrated confidence is usable: 93% confidence corresponds to 97% accuracy
rohanpaul_ai · x · 2026-11-06
Rohan Paul highlights that Drex models return calibrated probabilities: when the model says it's 70% sure, it's right about 70% of the time, and Drex Max is right 97% of the time when at least 93% confident.
That makes the confidence number usable as a "trust dial" — software can act autonomously on high-confidence outputs and route low-confidence ones to humans, something a chatbot saying "I'm fairly sure" can't offer.
The calibration chart also shows answer consistency: randomly reordering answer options flips only 4.3% of Max's answers, and the flipped ones are mostly cases the model already flagged as unsure.
More from Models
- Meituan's LongCat-2.5 multimodal model free for a week on Nous Portal — alexcovo_eth · 2026-11-07
- 8 Types of LLMs Powering AI Agents, from GPT Backbones to LAM Action Models — mdancho84 · 2026-11-07
- ChatGPT Images 2.6 vs Nano Banana 2.1: 4B weekly images vs a 50% price cut — Aiden_Tech_Ai · 2026-11-07
- Benchmarks: Kumo, Nori, Chronos-2 vs tuned LightGBM for power prices — nkafr · 2026-11-07
- a16z: AI Apps Shouldn't Price Per Token — Price at the Highest Value Layer — aleximm · 2026-11-07
- Ex-Rockstar Tech Director: Threat of AI Decompilation Ports May Force a Faster GTA VI PC Release — Ars Technica AI · 2026-11-07