# You Only Need One Model for Open-domain Question Answering

**Authors:** Haejun Lee, Akhil Kedia, Jongwon Lee, Ashwin Paranjape, Christopher D. Manning, Kyoung-Gu Woo  
**arXiv:** [2112.07381](https://arxiv.org/abs/2112.07381)  
**Date:** 2021-12-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2112.07381v1/

## Key points

- The paper proposes a new approach to Open-domain Question Answering (QA) using a singular model architecture instead of the traditional three-model approach.
- The existing approach involves separate retriever, reranker, and reader models with weakly coupled parameters during training.
- The proposed method uses hard-attention mechanisms within its transformer architecture to sequentially apply the retriever and reranker and feed resulting computed representations to the reader.
- This singular model architecture progressively refines hidden representations from the retriever to the reranker to the reader, leading to better gradient flow when trained in an end-to-end manner.
- A pre-training methodology is proposed to effectively train this architecture.
- The authors evaluate their model on Natural Questions and TriviaQA open datasets and show that their approach outperforms previous state-of-the-art models by 1.0 and 0.7 exact match scores for a fixed parameter budget.
- Contributions of this paper include proposing a new singular model architecture for Open-domain QA that efficiently uses model capacity while improving performance over previous approaches, utilizing hard attention mechanisms within its transformer architecture which enables end-to-end training with improved gradient flow compared to traditional approaches, and proposing a pre-training methodology which further boosts its performance on open domain QA tasks such as Natural Questions and TriviaQA datasets where it outperforms existing state of art models by 1.0 and 0.7 exact match scores respectively for fixed parameter budget.

## Summary

The paper titled "You Only Need One Model for Open-domain Question Answering" proposes a novel approach to Open-domain Question Answering (QA) that utilizes a singular model architecture instead of the traditional three-model approach. The existing approach involves using a retriever model to refer to an external knowledge base, optionally reranking passages with a separate reranker model, and generating an answer using another reader model. However, these models have separate parameters and are weakly coupled during training. The proposed method casts the retriever and the reranker as hard-attention mechanisms applied sequentially within the transformer architecture and feeds the resulting computed representations to the reader. This singular model architecture progressively refines hidden representations from the retriever to the reranker to the reader, leading to better gradient flow when trained in an end-to-end manner. Additionally, a pre-training methodology is proposed to effectively train this architecture. The authors evaluate their model on Natural Questions and TriviaQA open datasets and show that their approach outperforms previous state-of-the-art models by 1.0 and 0.7 exact match scores for a fixed parameter budget. The paper's contributions include proposing a new singular model architecture for Open-domain QA that efficiently uses model capacity while improving performance over previous approaches. This approach utilizes hard attention mechanisms within its transformer architecture which enables end-to-end training with improved gradient flow compared to traditional approaches. Furthermore, it also proposes a pre-training methodology which further boosts its performance on open domain QA tasks such as Natural Questions and TriviaQA datasets where it outperforms existing state of art models by 1.0 and 0.7 exact match scores respectively for fixed parameter budget.