Jev Fails Probability Calibration: Says 99/1 When the Answer Is 60/40
JnBrymn · x · 2026-09-19
Engineer JnBrymn stress-tested Jev's probability calibration: for the classic unfair-coin question with a 60/40 answer, Jev returned 99/1. Rephrasing the prompt from choice to noul makes the output at least sane, though the max error on this question with a known answer is still 9%. Full request/response examples included.
Related event: Jev model flunks probability calibration test, says 60/40 coin is 99/1(2 posts)→
More from Models
- RL-trained classification model Laya trends on Hugging Face — convaiinnovations · 2026-09-19
- OpenAI adds new voices to GPT-Live API, available in developer playground — juberti · 2026-09-19
- DeepSeek v4.1 flash paper figure shows sharp quality jump at 1M-token context — andrew_n_carr · 2026-09-19
- User charged $1,600 for idle containers on OpenAI's new Agents API — Wide-Arugula3042 · 2026-09-19
- Benchmarks show vision models spot image patterns but fail to pick the correct next image — AndrewDai · 2026-09-19
- Codex user burns through monthly quota 4 days early, complains pricey Astra tier offers no reset — NYCounihan · 2026-09-19