# Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots

**Authors:** Shaojie Jiang, Maarten de Rijke  
**arXiv:** [1809.01941](https://arxiv.org/abs/1809.01941)  
**Date:** 2018-09-06  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1809.01941v1/

## Key points

- Seq2Seq models have a low diversity issue in generating dialogue responses
- Diversity is important for engaging and varied responses in conversational settings
- The paper reviews known sources and approaches related to low-diversity issues
- Model over-confidence is identified as a significant source of low diversity
- Proposed solutions include confidence penalties and label smoothing techniques
- These methods aim to introduce uncertainty and encourage exploration of alternative response options
- The paper offers insights into the challenges posed by low diversity in Seq2Seq-based dialogue systems

## Summary

In their paper titled "Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots," authors Shaojie Jiang and Maarten de Rijke discuss the issue of low diversity in dialogue responses generated by sequence-to-sequence (Seq2Seq) models. While diversity has long been studied in information retrieval, it is also crucial in conversational settings where response generation systems should be able to produce engaging and varied responses. The authors acknowledge that Seq2Seq models have proven to be highly effective for response generation. However, they highlight the tendency of these models to generate responses with low diversity. To address this problem, the paper reviews known sources and existing approaches related to low-diversity issues. One significant source of low diversity that has received little attention so far is model over-confidence. The authors identify this as a potential cause for the lack of variation in dialogue responses. They propose several directions for tackling model over-confidence and subsequently addressing the low-diversity problem. Among the suggested approaches are confidence penalties and label smoothing techniques. These methods aim to introduce uncertainty into the model's predictions, encouraging it to explore alternative response options and generate more diverse outputs. Overall, this paper provides insights into the challenges posed by low diversity in Seq2Seq-based dialogue systems. By identifying model over-confidence as an overlooked factor contributing to this issue, the authors offer potential solutions that can enhance response generation systems' ability to produce diverse and interesting outputs.