# Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning

**Authors:** Justus-Jonas Erker, Gerasimos Spanakis, Stefan Schaffer  
**arXiv:** [2211.07591](https://arxiv.org/abs/2211.07591)  
**Date:** 2022-11-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.07591v1/

## Key points

- The paper introduces Curved Contrastive Learning (CCL) for abstract sequence modeling in dialogue planning.
- CCL utilizes contrastive learning sentence embeddings to generate semantically meaningful and conversational graph curved utterance embeddings.
- Bi-encoder models are proposed to guide transformers as response ranking models in a zero-shot fashion.
- Cosine similarity in the curved space indicates the distance/reachability of a candidate utterance towards the corresponding goal.
- Forward-entailing language representations can be used to predict future patterns in dialogues by measuring entailment strength through cosine similarity.
- Conversations from the DailyDialog dataset show strong evidence of planning capability over multiple turns, with 61.56% accuracy over 3 turns.
- CCL allows for faster ranking of one million utterance & context pairs compared to DialogRPT, with GPU computation time over 7 million times faster.
- CCL is on average 2.8% qualitatively superior for sequences longer than 2 turns.
- Authors: Justus-Jonas Erker, Gerasimos Spanakis, Stefan Schaffer.

## Summary

The paper titled "Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning" introduces a technique called Curved Contrastive Learning (CCL) that utilizes contrastive learning sentence embeddings to generate semantically meaningful and conversational graph curved utterance embeddings. These embeddings can be compared using cosine similarity. The authors propose bi-encoder models that guide transformers as response ranking models in a zero-shot fashion by projecting the goal utterance and corresponding reply candidates into a latent space. The cosine similarity in this curved space indicates the distance/reachability of a candidate utterance towards the corresponding goal. The paper also explores how these forward-entailing language representations can be used to assess the likelihood of sequences by measuring the entailment strength through cosine similarity of individual members encoded separately in the curved space. This allows for predicting future patterns in dialogues, specifically by ordering/identifying future goal utterances that are multiple turns away, given a dialogue context. To analyze the characteristics that make conversations (un)plannable, the authors investigate conversations from the DailyDialog dataset and find strong evidence of planning capability over multiple turns, with 61.56% accuracy over 3 turns. Additionally, the paper demonstrates how the curved property can be exploited to rank one million utterance & context pairs much faster than DialogRPT, with GPU computation time over 7 million times faster. Moreover, it shows that CCL is on average 2.8% qualitatively superior for sequences longer than 2 turns. The authors of this paper are Justus-Jonas Erker, Gerasimos Spanakis, and Stefan Schaffer.