Calibrate Before Use: Improving Few-Shot Performance of Language Models
Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, Sameer Singh
ICML 2021
cs.CL, cs.LG
2021-02-19
On SST-2, GPT-3 few-shot accuracy swings 54.3% to 93.4% by reordering four examples. Contextual calibration with N/A probes lifts mean accuracy by up to 30.0 points.
GPT-3 can classify, fill slots, and complete facts from a handful of examples in the prompt, with no gradient step. The real problem is instability. Change the template, the examples, or even the order of those examples, and accuracy can jump from near chance to near the then-SOTA.
On SST-2 with GPT-3 2.7B and four shots, permuting the same four examples moves accuracy from 54.3% to 93.4%. In the appendix, swapping two examples drops 88.5% to 51.3%. Variance stays high at 16 shots. On DBPedia, mean accuracy falls from 36.0% zero-shot to 25.9% one-shot. Anyone writing prompts by hand cannot count on a stable number.
Three biases shift the output distribution as a whole.
Majority label bias: the model repeats whichever answer is more common in the prompt. On 4-shot LAMA with GPT-3 2.7B, 50.2% of predictions copy one of the four training answers; the true repeat rate is 24.7%. The common 0-shot to 1-shot drop is often the model parroting the single label it just saw.
Recency bias: answers nearer the end of the prompt get copied more. Four sentiment examples ordered positive-positive-positive-negative yield nearly 90% Negative predictions on a balanced validation set, even though three of four shots are Positive. On LAMA, the model over-predicts the 1st through 4th training answers by 8.5%, 8.3%, 14.3%, and 16.1%. Recency can overpower majority.
Common token bias: the model prefers tokens that were frequent in pretraining. On balanced 14-way DBPedia it predicts "book" 11 times as often as "artist". Label frequency in Google Ngrams correlates with prediction rate at r=0.67.
Together these biases often just slide the decision boundary. For one SST-2 prompt, replacing the test sentence with "N/A" yields 61.8% Positive. The default 50% threshold then produces many false positives. An oracle threshold at 0.68 would hit 94% on the validation set.
Calibration therefore needs no extra labels. Feed a content-free probe through the same prompt (the paper averages "N/A", "[MASK]", and the empty string) to get a prior pcf. For classification, set W = diag(pcf)^{-1} and b = 0, which rescales so the dummy input is uniform across classes. For generation the vocabulary is about 50,000, so a diagonal W is unwieldy; they set b = -pcf and W to identity, and only calibrate the first token. One extra forward pass, a few lines of code.
"Calibration" here means zeroing an instrument, not Guo et al.'s statistical calibration of confidence to accuracy. The OpenAI API then exposed probabilities, not logits, so the affine map sits after the softmax.
Fixed template, five random draws of 0/1/4/8-shot examples, labels left unbalanced. Baseline is Brown et al. greedy decoding.
| Setting | Baseline | Calibrated |
| AGNews, GPT-3 175B, 0-shot | 43.9 | 73.9 |
| DBPedia, GPT-3 175B, 0-shot | 22.0 | 59.7 |
| SST-2, GPT-3 2.7B, 8-shot | 54.0 | 82.0 |
| AGNews, GPT-3 2.7B, 4-shot | 43.3 | 71.1 |
| TREC, GPT-3 175B, 1-shot | 57.7 | 75.7 |
| DBPedia, GPT-3 2.7B, 1-shot | 25.9 | 61.6 |
The largest absolute gain is 30.0 points, on AGNews with GPT-3 175B zero-shot. A calibrated 2.7B model sometimes beats an uncalibrated 175B baseline, by as much as 19.3 points, at about 1/50 the size. Variance across training sets falls on most tasks. Of four 0-shot to 1-shot drops, calibration removes three.
Gains are not universal. On RTE, GPT-3 175B is flat at 57.8 zero-shot and slips from 66.2 to 65.5 at 8-shot. LAMA mainly moves in the zero-shot column (2.7B 14.0 to 22.7, 175B 23.5 to 30.1); 4-shot and 8-shot barely change. GPT-2 1.5B shows the same instability and takes the same fix.
The dummy string matters, but many work. On 1-shot SST-2 / 0-shot AGNews: baseline 66.5 / 48.5, "N/A" 74.2 / 64.5, the three-input ensemble 79.0 / 66.5, gibberish tokens 79.3 / 64.5. On AGNews, this data-free procedure already sits close to an oracle that searches the best diagonal W on the validation set.
This is an early, unusually clear failure-mode-plus-patch paper for in-context learning. Part of prompt engineering becomes a measurable output shift: estimate the prior on a blank input, then divide it out. For classification, multiple choice, and closed-set slot filling, if class token probabilities are available, diagonal rescaling is still a cheap default post-process. Cost is one extra forward pass and no labels.
It does not retire prompt engineering. The authors say calibration pulls best, average, and worst prompts closer together and higher, but the gap remains. On open-weight models, logits are cleaner than API probabilities, so the affine map should sit before softmax. For open-ended generation, chain-of-thought, and tool use, first-token calibration is not enough.
A second lesson for anyone reading few-shot papers: a single prompt's point estimate can be propped up by example order. The authors recommend putting contextual calibration into probing and evaluation, otherwise model accuracy is understated.
Calibration is not always a win. RTE barely moves. LAMA saturates once examples are present. On ATIS departure date with GPT-3 13B one-shot, the mean even falls (97.9 to 95.5). Content-free strings were chosen on AGNews and reused everywhere; a task-specific probe appears only for LAMA ("N/A was born in").
The labeling budget stops at 8 shots because the OpenAI API was expensive. Figure 1 plots 16-shot variance; the full calibration table does not. Mapping probabilities rather than logits is an API constraint, not an optimal design. Generation calibrates only the first token; open-ended generation is left as future work.
External validity needs a discount. These are 2021 GPT-3 and GPT-2. Instruction tuning, RLHF, and chat templates change how majority and recency biases show up. Dropping "N/A" into a modern chat wrapper may not estimate the prior you think it does. The paper also never checks whether the adjusted probabilities are statistically calibrated. The word "calibrate" in the title is easy to misread as confidence alignment.