# Copy Is All You Need

**Authors:** Tian Lan, Deng Cai, Yan Wang, Heyan Huang, Xian-Ling Mao  
**arXiv:** [2307.06962](https://arxiv.org/abs/2307.06962)  
**Date:** 2023-07-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.06962v1/

## Key points

- COG (Copy is All You Need) is a novel approach for text generation
- COG generates text by copying meaningful text segments from an existing collection
- Contextualized representations of the segments are computed and indexed using efficient vector search toolkits
- COG outperforms other models in terms of generation quality on the WikiText-103 benchmark dataset, as evaluated by automatic metrics and human evaluations
- COG demonstrates comparable inference efficiency to token-level autoregressive models due to reduced decoding steps
- COG can adapt to different domains without additional training by switching to a domain-specific text collection
- Scaling up to larger text collections leads to performance gains without further training
- COG integrates retrieval into the generation process itself, unlike previous approaches that combine retrieval and generation separately

## Summary

In this paper, the authors propose a novel approach called COG (Copy is All You Need) for text generation. Unlike traditional models that select words from a fixed vocabulary, COG generates text by progressively copying meaningful text segments from an existing collection. The authors compute contextualized representations of these segments and index them using efficient vector search toolkits. This allows for a series of copy-and-paste operations during text generation, where suitable text spans are selected from the collection instead of a standalone vocabulary. The authors conducted experiments on the WikiText-103 benchmark dataset and found that COG outperforms other models in terms of generation quality, as evaluated by both automatic metrics and human evaluations. Additionally, COG demonstrates comparable inference efficiency to token-level autoregressive models due to the reduction in decoding steps. One notable advantage of COG is its ability to adapt to different domains without additional training. By simply switching to a domain-specific text collection, COG can effectively generate text in that domain. Furthermore, the authors observed that scaling up to larger text collections also leads to performance gains without requiring further training. The paper also discusses related work in retrieval-augmented text generation and highlights how COG differs from prior approaches. While previous work combines retrieval and generation processes separately, COG integrates retrieval into the generation process itself. Overall, the experimental results support the effectiveness of COG in generating high-quality text by leveraging existing collections. The authors acknowledge valuable suggestions from reviewers that led to revisions in their experiments and express gratitude for improving the quality of their paper.