Are You Sure You're Sure? On the Impact of Instruction Tuning on Confidence and Lexical Diversity
Irina Proskurina, Mayank Kumar, Oyindolapo O. Komolafe
cs.CL, cs.AI
2026-08-14
Across Qwen2.5-7B, Mistral-7B, and Llama-3.1-8B pairs, instruction tuning consistently raises verbalized confidence (e.g. 49.2% to 90.4% with unchanged accuracy) while shrinking cross-rationale diversity, and the two shifts are statistically unrelated.
Instruction-tuned language models have been repeatedly observed to be verbally overconfident: their self-reported confidence far exceeds their accuracy. This paper asks a sharper question: when instruction tuning shifts confidence, what happens to the lexical makeup of the reasoning chains (rationales) the model generates along the way? If confidence rises while rationales narrow, overconfidence is not just a calibration issue but ties into a loss of generation diversity. The team is from Cohere Labs Community with collaborators in France, India, and Nigeria.
Three matched pairs, Qwen2.5-7B, Mistral-7B-v0.3, and Llama-3.1-8B, each in base and instruct versions, answering ARC-Easy, MMLU, and CommonsenseQA, with 5 rationales sampled per question via chain-of-thought prompting. Confidence is measured three ways: entropy over candidate answer distribution, verbalized confidence from a two-pass "how sure are you" prompt, and likelihood-based ECE. Diversity splits into two measures: Unique-2 (proportion of distinct bigrams, the lexical richness within one rationale) and 1-SelfBLEU (how different each rationale is from the other four, i.e. cross-rationale variability). A controlled analysis keeps only questions where base and instruct variants pick the same answer with matched rationale lengths.
Confidence moves in one direction everywhere: across all nine model-by-benchmark cells, answer entropy drops significantly and verbalized confidence rises significantly, while accuracy barely moves. The starkest case is Llama-3.1-8B on ARC-Easy: accuracy flat at 82.2%, verbalized confidence jumping from 49.2% to 90.4%. Mistral on CSQA climbs from 47.9% to 92.2% verbalized confidence alongside an 11.8-point accuracy gain. Likelihood-based ECE worsens, and its direction does not track diversity changes.
Diversity splits by measure:
| Measure | Effect of instruction tuning |
| Unique-2 (within-chain richness) | Direction varies; Mistral drops on ARC-Easy but rises on CSQA |
| 1-SelfBLEU (cross-chain dissimilarity) | Decreases in all nine cells, no exceptions |
The split survives controls: on the length-matched CSQA subset, Mistral and Llama show significantly higher Unique-2 (+0.050/+0.053) alongside significantly lower 1-SelfBLEU (-0.069/-0.012). Individual rationales use more varied words while the five chains per question read more like retellings of each other.
Systems that estimate uncertainty from sampling consistency should take note: higher verbalized confidence does not mean the model is more certain, since accuracy is flat and ECE worsens. What rose is posture, not capability. The uniform shrink in cross-rationale diversity means self-consistency voting gets less information per sample: as chains converge toward paraphrases of one another, more samples amount to double counting. Applications using rationale diversity as an uncertainty proxy on tuned models need recalibration.
The authors list several: three English multiple-choice benchmarks only; diversity measured lexically, not semantically or syntactically; no cross-lingual check; and no safety- or demographic-sensitive prompts, so nothing here extends to harmful outputs. Reading the paper adds a few: only 5 rationales per question is a small sample for cross-chain estimates; the instruct training data and recipes of the three vendors are opaque, so attribution to instruction data is impossible; and the whole analysis is correlational, with no causal direction between confidence and diversity. The novelty over prior work on instruction-tuning overconfidence sits mainly in the careful decomposition of the diversity side.