Dice tests expose LLM calibration failures

Calibration tests with fair dice showed Jev reporting 83% confidence while achieving only 19% accuracy; discussants argued calibration itself is the product, since demos hide the cost of wrong decisions.

2026-09-24 ~ 2026-09-25 · 2 related posts

• The coin-flip test: why LLMs fail at calibration, which is the actual product — airesearch12 · 2026-09-24
• Jev Does Not Play Dice: 83% confidence, 19% accuracy on a fair die roll — kh-ai · 2026-09-25