Cohere's CALIBER cuts LLM calibration error 52.5% by training pre- and post-reasoning confidence separately
Cohere_Labs · x · 2026-10-20
Cohere Labs introduces CALIBER, arguing confidence in reasoning models is state-dependent: pre-thinking confidence should estimate the chance of solving a prompt, while post-answer confidence should predict whether the realized answer is correct — each supervised with a matched target. On BigMathDigits, CALIBER cuts Expected Calibration Error by 52.5% over the strongest single-confidence baseline for a 7B model while achieving the best Brier score and AUROC within 2.1 points of best accuracy; a 30B model also gets the best ECE, and results hold out-of-distribution on GPQA and TriviaQA.
More from Research
- Anima Anandkumar on neural operators: AI weather models beat supercomputers — AnimaAnandkumar · 2026-10-23
- Scaling GLM-5.3-Flash training to 1M token context, 64x over Megatron baseline — AndrewLBeam · 2026-10-23
- Experimental boolean-logic LM: microsecond forward pass on one CPU thread, but painfully slow training — cephaloform · 2026-10-23
- AdaDex turns seconds of human hand motion into generalizable dexterous robot skills — mangahomanga · 2026-10-23
- ReAPO paper: conditioning retries on failures beats GRPO in RLVR reasoning training — jasondeanlee · 2026-10-23
- Quantum-JEPA: first JEPA world model deployable on real quantum circuits (QPU) — DrMorganLevine · 2026-10-23