# Conformal Prediction with Large Language Models for Multi-Choice Question Answering

**Authors:** Bhawesh Kumar, Charlie Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, Andrew Beam  
**arXiv:** [2305.18404](https://arxiv.org/abs/2305.18404)  
**Date:** 2023-05-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.18404v3/

## Key points

- Importance of robust uncertainty quantification techniques for safe deployment of large language models in high-stakes scenarios
- Use of conformal prediction to provide uncertainty estimates in language models for multiple-choice question-answering tasks
- Correlation between uncertainty estimates obtained from conformal prediction and prediction accuracy
- Implications for downstream applications such as selective classification and filtering out low-quality predictions
- Investigation of the exchangeability assumption required by conformal prediction for out-of-subject questions
- Experiments showing strong correlation between uncertainty estimates and prediction accuracy
- Potential applications of uncertainty estimates in selective classification and filtering out low-quality predictions
- Addressing the performance of conformal prediction with out-of-subject questions
- Contribution to ensuring trustworthy and reliable usage of large language models in safety-critical situations through robust uncertainty quantification techniques.

## Summary

This research focuses on the importance of robust uncertainty quantification techniques for the safe deployment of large language models in high-stakes scenarios. Specifically, the study explores how conformal prediction can be used to provide uncertainty estimates in language models for multiple-choice question-answering tasks. The researchers find that the uncertainty estimates obtained from conformal prediction are closely correlated with prediction accuracy. This finding has significant implications for downstream applications, such as selective classification and filtering out low-quality predictions. Additionally, the study investigates the exchangeability assumption required by conformal prediction for out-of-subject questions, which is a more realistic scenario in many practical applications. By addressing these challenges, this work contributes to ensuring more trustworthy and reliable usage of large language models in safety-critical situations where robust guarantees of error rate are essential. In terms of methodology, experiments were conducted to evaluate the effectiveness of conformal prediction in providing uncertainty quantification for multiple-choice question-answering tasks. The results showed a strong correlation between uncertainty estimates and prediction accuracy, indicating that conformal prediction can effectively capture uncertainties in language models' responses. Furthermore, potential applications of these uncertainty estimates were explored. Selective classification and filtering out low-quality predictions were identified as valuable downstream applications that can benefit from leveraging the tight correlation between uncertainty estimates and prediction accuracy. The researchers also addressed an important aspect related to out-of-subject questions by investigating the exchangeability assumption required by conformal prediction when dealing with questions outside the model's training domain. This scenario is more representative of real-world applications where language models may encounter unfamiliar topics or subjects. By examining this assumption, the study provided insights into how conformal prediction performs when faced with out-of-subject questions. Overall, this research contributes to enhancing the trustworthiness and reliability of large language models in safety-critical situations by developing robust uncertainty quantification techniques through conformal prediction. The findings highlight the potential applications of these techniques in selective classification and filtering out low-quality predictions. Additionally, the investigation of the exchangeability assumption expands the understanding of how conformal prediction can handle out-of-subject questions, making it more applicable to practical scenarios.