# CiteBench: A benchmark for Scientific Citation Text Generation

**Authors:** Martin Funkquist, Ilia Kuznetsov, Yufang Hou, Iryna Gurevych  
**arXiv:** [2212.09577](https://arxiv.org/abs/2212.09577)  
**Date:** 2022-12-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.09577v1/

## Key points

- Rapidly evolving landscape of scientific research
- Growing interest in automating summarizing and synthesizing research papers
- Introduction of CiteBench benchmark for standardized evaluation of citation text generation models
- Exploration of performance of strong baselines and transferability between datasets
- Common task formulation and evaluation framework through CiteBench
- Advancement in automated literature review and promotion of collaboration within the scientific community

## Summary

, , , , 

In the rapidly evolving landscape of scientific research, keeping up with the latest findings has become increasingly challenging. To address this issue, there is a growing interest in automating the process of summarizing and synthesizing research papers in various fields. One specific area of focus is citation text generation, where models are tasked with generating citation texts based on a set of cited papers and the context provided by the citing paper. Previous studies on citation text generation have utilized diverse datasets and task definitions, leading to a lack of standardization in evaluating and comparing different models. In response to this challenge, the authors introduce CiteBench: a benchmark designed to unify existing datasets and facilitate standardized evaluation of citation text generation models across different domains and task settings. By leveraging this new benchmark, the authors explore the performance of several strong baselines, assess their transferability between datasets, and offer valuable insights into task definition and evaluation practices that can guide future research in citation text generation. Furthermore, previous research has approached citation text generation from various perspectives, including extractive or abstractive summarization methods, single or multiple cited papers input, as well as outputting either a single sentence or a paragraph as the generated citation text. Despite these advancements, the lack of a common task formulation and evaluation framework has hindered direct comparisons between different studies. Through CiteBench, researchers now have access to a unified platform that consolidates four existing datasets for citation text generation tasks. Overall, this work not only contributes to advancing the field of automated literature review but also sets a foundation for more systematic and comprehensive evaluations in citation text generation research. The availability of CiteBench as an open-source resource further promotes collaboration and knowledge sharing within the scientific community.