# Shifting Attention to Relevance: Towards the Uncertainty Estimation of Large Language Models

**Authors:** Jinhao Duan, Hao Cheng, Shiqi Wang, Chenan Wang, Alex Zavalny, Renjing Xu, Bhavya Kailkhura, Kaidi Xu  
**arXiv:** [2307.01379](https://arxiv.org/abs/2307.01379)  
**Date:** 2023-07-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.01379v1/

## Key points

- Research conducted by Jinhao Duan et al. focuses on characterizing uncertainty in model generations produced by Large Language Models (LLMs) for Natural Language Generation.
- Study reveals generative inequalities in reflecting the meaning of generations by auto-regressive LLMs, where some tokens are more relevant than others.
- Tokens are equally valued during uncertainty estimation due to linguistic redundancy, even though not all tokens carry equal importance.
- Proposed approach called Shifting Attention to more Relevant (SAR) components aims to address biases from generative inequalities at both token and sentence levels during uncertainty estimation.
- Experiments involved popular LLMs like OPT and LLaMA, as well as commercial LLMs like Davinci from OpenAI, across various free-form question-answering tasks.
- SAR demonstrated superior performance in mitigating biases caused by generative inequalities, as shown through experimental outcomes and detailed demographic analysis.
- This research provides valuable insights for improving uncertainty estimation in Large Language Models for more reliable model outputs in Natural Language Generation tasks.

## Summary

The research conducted by Jinhao Duan, Hao Cheng, Shiqi Wang, Chenan Wang, Alex Zavalny, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu delves into the challenges of characterizing uncertainty in model generations produced by Large Language Models (LLMs) for Natural Language Generation. The study highlights the unequal creation of tokens in reflecting the meaning of generations by auto-regressive LLMs. It is found that some tokens are more relevant or representative than others. However, all tokens are equally valued when estimating uncertainty due to linguistic redundancy where a few keywords can convey the meaning of a long sentence. These inequalities are termed generative inequalities and are investigated for their impact on uncertainty estimation. The results of the research reveal that significant tokens and sentences with limited semantics are weighted equally or even heavily during uncertainty estimation. To address these biases stemming from generative inequalities, the authors propose a novel approach called Shifting Attention to more Relevant (SAR) components at both the token and sentence levels while estimating uncertainty. The experiments conducted involve popular "off-the-shelf" LLMs such as OPT and LLaMA with model sizes up to 30B as well as powerful commercial LLMs like Davinci from OpenAI across various free-form question-answering tasks. The experimental outcomes and detailed demographic analysis demonstrate the superior performance of SAR in mitigating biases caused by generative inequalities. This research contributes valuable insights towards improving uncertainty estimation in Large Language Models for more reliable model outputs in Natural Language Generation tasks. The code for implementing this approach is made available at https://github.com/jinhaoduan/shifting-attention-to-relevance.