Search Results for « Prediction-Powered Inference »

Prediction-powered inference (PPI) combines a small gold-standard labeled sample with a large unlabeled sample scored by a black-box predictor, correcting the predictor's bias through a rectifier measured on labeled data. We recast PPI as posterior transport. The missing labels on the unlabeled set are treated as latent quantities and imputed by a learned conditional generative map, and the posterior over the estimand is the pushforward of a base measure through that map and the estimating equation. This is the Generative Bayesian Computation (GBC) construction applied to a missing-label problem. The additive rectifier of classical PPI, the power-tuning parameter of PPI++, Rubin's multiple imputation, and the augmentation term of doubly-robust estimation all appear as reductions of a single transport map: classical PPI is the feature-free mean-shift map, PPI++ is a shrinkage of the map toward the surrogate, multiple imputation is the map itself with a proper between-map variance, and the augmented inverse propensity weighted estimator is the Bayesian analogue with a labeling indicator whose propensity is known by design. Unlike linear corrections, the learned map captures nonlinear and heteroskedastic surrogate-totruth structure directly, so the stratified and filtered variants of PPI become choices of map class rather than separate methods. We give the transport formulation, prove the reductions, connect the construction to the semiparametric efficiency bound of the model that conditions on the surrogate, and provide an algorithm. We are explicit about the three places where the method can fail: honest extrapolation beyond labeled support, correct propagation of map uncertainty for calibration, and the price of a rich map relative to the PPI++ efficiency guarantee. When the map class contains the affine map, GBC-PPI inherits the guarantee that it is asymptotically no worse than the labeled-only estimator.

Prediction-powered inference (PPI) performs valid inference when a small gold-standard labeled sample is supplemented by a large pool of machine-learning predictions. Naive substitution of predictions for outcomes changes the estimand and invalidates uncertainty quantification; the shared remedy is rectification, in which the abundant predictions supply a lowvariance plug-in while the scarce labels estimate and correct the resulting distortion. We review PPI as a family of rectified M-estimators organised by the single object every method manipulates: the rectifier, the labeled-data estimate of the surrogate's inferential bias. Four operations on the rectifier organise the field. One may tune it by an optimal, generally matrix-valued gain (PPI++, cross-PPI, and the Bayes-linear form that unifies them); reweight it for informative or active label acquisition (inverse-probability weighting, active and stratified sampling); shrink it with a prior (frequentist-assisted-by-Bayes and Bayesian PPI, through global-local scale mixtures); or replace it by a recalibration model (Efron's bootstrap). We give a self-contained derivation of the tuned estimator's efficient influence function, from which the optimal gain, its equivalence to a Kalman gain, and the semiparametric efficiency bound follow; we prove that the failure of PPI to improve estimation of E{Y } is exactly the corner of optimal tuning; and we show that horseshoe shrinkage of the rectifier is a scale-mixture Kalman update rather than a special case of the linear form. A combined design-weighted, global-local estimator is proposed that is consistent under informative labeling and strictly tighter than inverse-probability-weighted PPI when the surrogate is well calibrated. A five-part simulation study delineates the regime in which each operator dominates, and a decision guide closes the paper.

Prediction-powered inference (PPI) studies statistical inference when a small set of gold-standard labels is combined with a much larger pool of machine-generated predictions. The central difficulty is that predictions can substantially reduce variance, yet naive substitution of predictions for outcomes generally changes the estimand and invalidates uncertainty quantification. The basic remedy in the literature is rectification: predictions are used to construct a low-variance plug-in term, while labeled observations are used to estimate and correct the inferential distortion induced by prediction substitution. We review PPI as a family of rectified plug-in procedures for hybrid measurement regimes. The survey develops a common statistical template based on mean estimation, estimating equations, and loss-based formulations, and then uses that template to compare modern variants according to the component they modify: the rectification engine, the label-acquisition design, predictor dependence, or the validity target. We also position PPI relative to model-assisted survey sampling, post-prediction correction, surrogate-outcome methods, classical measurement-error models, and semiparametric augmentation. Throughout, we distinguish questions of validity from questions of efficiency, robustness, and computation, and we emphasize that valid use of prediction assistance does not require a correct predictive model but does depend on how rectification, dependence, and sampling design are handled. The survey closes with recurrent failure modes, practical reporting recommendations, and open problems in finite-sample theory, heterogeneous proxy quality, and protocol-aware deployment.

While reliable data-driven decision-making hinges on high-quality labeled data, the acquisition of quality labels often involves laborious human annotations or slow and expensive scientific measurements. Machine learning is becoming an appealing alternative as sophisticated predictive techniques are being used to quickly and cheaply produce large amounts of predicted labels; e.g., predicted protein structures are used to supplement experimentally derived structures, predictions of socioeconomic indicators from satellite imagery are used to supplement accurate survey data, and so on. Since predictions are imperfect and potentially biased, this practice brings into question the validity of downstream inferences. We introduce cross-prediction: a method for valid inference powered by machine learning. With a small labeled dataset and a large unlabeled dataset, cross-prediction imputes the missing labels via machine learning and applies a form of debiasing to remedy the prediction inaccuracies. The resulting inferences achieve the desired error probability and are more powerful than those that only leverage the labeled data. Closely related is the recent proposal of prediction-powered inference [A. N. Angelopoulos, S. Bates, C. Fannjiang, M. I. Jordan, T. Zrnic, Science 382 , 669–674 (2023)], which assumes that a good pretrained model is already available. We show that cross-prediction is consistently more powerful than an adaptation of prediction-powered inference in which a fraction of the labeled data is split off and used to train the model. Finally, we observe that cross-prediction gives more stable conclusions than its competitors; its CIs typically have significantly lower variability.

Prediction-powered inference (PPI) and its subsequent development called PPI++ provide a novel approach to standard statistical estimation leveraging machine learning systems to enhance unlabeled data with predictions. We use this paradigm in clinical trials. The predictions are provided by disease progression models, providing prognostic scores for all the participants as a function of baseline covariates. The proposed method would empower clinical trials by providing untreated digital twins of the treated patients while remaining statistically valid. The potential implications of this new estimator of the treatment effect in a two-arm randomized clinical trial (RCT) are manifold. First, it leads to an overall reduction of the sample size required to reach the same power as a standard RCT. Secondly, it advocates for an imbalance of controls and treated patients, requiring fewer controls to achieve the same power. Finally, this technique directly transfers any disease prediction model trained on large cohorts to practical and scientifically valid use. In this paper, we demonstrate the theoretical properties of this estimator and illustrate them through simulations. We show that it is asymptotically unbiased for the Average Treatment Effect and derive an explicit formula for its variance. An application to an Alzheimer's disease clinical trial showcases the potential to reduce the sample size.

Missing covariate data is a prevalent problem in empirical research. We provide a novel framework for handling missing covariate data for estimation and inference in downstream tasks. Our general framework provides an automatic and easy-to-use pipeline for empirical researchers: First, missing values are imputed using virtually any imputation method under general observation patterns. Second, we automatically correct for the imputation bias and adaptively weight the imputed values according to their quality. Third, we use all available data, including imputed observations, to obtain more precise point estimates for the downstream task with valid confidence intervals. Our approach ensures valid inference while improving statistical efficiency by leveraging all available data. We establish the asymptotic normality of the proposed estimator under general missing data patterns and a broad class of imputation methods. Through simulations, we demonstrate the superior performance of our approach over natural benchmarks, as it achieves both lower bias and variance while being robust to imputation quality. In a comprehensive empirical study of the dependence of equity markets on carbon emissions, we show that properly accounting for missing emissions data yields no evidence of correlation between stock returns and emissions directly produced by companies, but a negative correlation with value chain emissions.<br><br>Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at <a href="http://www.nber.org/papers/w34535" TARGET="_blank">www.nber.org</a>.<br>