2026-09-13
Stanford ties PPI to surrogate outcomes and proposes Recalibrated PPI, using ~19% fewer labels than PPI++ on wine reviews and ~5% fewer on X-rays and toxicity.
True outcomes are expensive. Surrogates are cheap. Biostatistics and economics have used that bargain for thirty years: cardiac enzymes for infarct damage, CD4 counts for HIV progression, short-run employment for long-run earnings. The expensive object now is human labels. The cheap object is a model prediction. Prediction-powered inference (PPI) treats outputs from pretrained models as this new class of surrogate.
Predictions and outcomes often live on different scales. A chest X-ray classifier never sees age. A toxicity model trained on Wikipedia is dropped onto news comments. A language model collapses wine reviews into five discrete bins. Plug Ŷ in for Y and the interval can be wider than the labeled-only estimator. Ji, Lei, and Zrnic at Stanford reconnect PPI to the classical surrogate-outcome model and introduce Recalibrated PPI (RePPI).
The PPI template is empirical risk on labeled data, minus the gap between labeled and unlabeled imputed losses. A good imputed loss turns unlabeled predictions into a control variate and cuts variance. A bad one inflates it.
RePPI adds a recalibration step. Flexible machine learning estimates the optimal imputed score E[∇ℓθ(X,Y) | X, Ŷ] on labeled data, then that score builds the imputed loss. Three-fold cross-fitting keeps the initial estimator, the score fit, and the inferential step on separate folds, so overfitting cannot split the labeled and unlabeled score distributions.
Two guarantees sit at the center. If the optimal imputed loss is estimated consistently, RePPI matches the PPI-family efficiency bound. If it is not, the estimator still beats the XY-only estimator, because the correction matrix M is chosen from sample covariances to force an improvement. The objective stays convex whenever the original loss is convex. Perfect predictions make RePPI equivalent to ERM on n+N samples. If Ŷ is just a function of X, the prediction itself adds no information, yet recalibrating on X can still shave variance.
Three stylized settings where predictions systematically miss the outcome, plus three real datasets, labeled fractions 10% to 30%, against XY-only, PPI, and PPI++.
Chest X-rays (NIH ChestX-ray14, 29,366 cases): DenseNet-121 sees only the image; the target is the logistic coefficient of standardized age on pleural effusion. Raw PPI intervals are so wide the figure drops the curve. To reach width 0.20, XY-only needs 7,704 labels, PPI++ 7,602, RePPI 7,164, a 5.74% cut versus PPI++.
Toxicity (10,000 CivilComments-WILDS rows): DistilBERT trained on Jigsaw Wikipedia comments; the target is the regression coefficient on identity mentions. PPI often loses to XY-only. At width 0.040 the counts are 2,075 / 1,847 / 1,739, a 5.85% cut.
Wine reviews (10,000 US wines): GPT-5.4 nano maps the tasting note to bins 1-5; the target is the price coefficient on rating. PPI++ can only apply a linear correction; RePPI fits a two-layer net. At width 0.33 the counts are 2,744 / 2,625 / 2,163, a 17.57% cut; other widths save about 18% to 21%.
An appendix census regression of log income on age, with the predictor trained only on college graduates and inference on the full population, saves more than 24% of labels versus PPI++. Coverage in these experiments sits near 95%.
| Task | Target width | PPI++ labels | RePPI labels | Saved |
| Chest X-ray age coefficient | 0.20 | 7602 | 7164 | 5.74% |
| News-comment toxicity | 0.040 | 1847 | 1739 | 5.85% |
| Wine price coefficient | 0.33 | 2625 | 2163 | 17.57% |
| Census wages (appendix) | 0.0023 | 10287 | 7594 | 26.18% |
Anyone using an LLM as a cheap annotator, then regressing on those labels, gets a variance formula instead of a slogan. The work is in mapping predictions onto the scale of the inferential target. A more accurate model does not close that gap by itself. Cross-modal X-rays and discrete wine bins are exactly where linear power tuning (PPI++) runs out; nonlinear recalibration is the extra step.
The method is usable: convex loss stays convex; cross-fitting keeps score learning off the inferential fold. Code is in the paper's repository. The scope is semi-supervised inference. It does not argue that a model can replace a clinical trial endpoint.
The authors list three. Unstructured context W enters only through Ŷ = f(X,W), so embeddings, logits, uncertainty scores, and multi-model outputs are thrown away. Guarantees are asymptotic; finite-sample behavior tracks how well the imputed score is learned, how many labels you have, and how strong the surrogate signal is. The main setting is i.i.d. labeled and unlabeled samples. Unlabeled data from a different source is handled in the appendix by density-ratio weights; a bad ratio estimate can inject first-order bias, and they leave a full analysis for later.
The evaluation is a bit thin in one place: the full-data estimate is used as a proxy for θ⋆ when scoring coverage, which can make coverage look cleaner in finite samples. On X-rays and toxicity the saving versus PPI++ is only about 5%, so the extra cross-fit is worth it only if a label is costly. Wine and census show larger gains, because discrete bins and shift sit in the blind spot of linear correction.