# Small Language Models are Good Too: An Empirical Study of Zero-Shot Classification

**Authors:** Pierre Lepagnol, Thomas Gerald, Sahar Ghannay, Christophe Servan, Sophie Rosset  
**arXiv:** [2404.11122](https://arxiv.org/abs/2404.11122)  
**Date:** 2024-04-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.11122v1/

## Key points

- Study titled "Small Language Models are Good Too: An Empirical Study of Zero-Shot Classification"
- Authors: Pierre Lepagnol, Thomas Gerald, Sahar Ghannay, Christophe Servan, Sophie Rosset
- Focus on evaluating performance of small language models in zero-shot text classification
- Challenge belief that larger models dominate in text classification
- Evaluation conducted on 15 datasets using language models ranging from 77 million to 40 billion parameters
- Findings show small language models can effectively classify texts and sometimes outperform larger models
- Development of open-source repository for transparency and further research
- Small language models can be resource-efficient solutions for specific data classification challenges
- Valuable insights provided for optimizing model size in text classification tasks

## Summary

In their study titled "Small Language Models are Good Too: An Empirical Study of Zero-Shot Classification," authors Pierre Lepagnol, Thomas Gerald, Sahar Ghannay, Christophe Servan, and Sophie Rosset delve into the ongoing debate surrounding the efficiency of large versus small language models for text classification through prompting. The research focuses on evaluating the performance of small language models in zero-shot text classification and challenges the belief that larger models dominate in this area. The investigation spans 15 datasets and involves benchmarking language models ranging from 77 million to 40 billion parameters. Various architectures and scoring functions are utilized in this comprehensive analysis. Surprisingly, the findings reveal that small language models can effectively classify texts and often perform on par with or even surpass their larger counterparts. This challenges the common notion that bigger models always yield better results. To facilitate further research and promote transparency, the authors have developed a comprehensive open-source repository that encapsulates their methodologies. This resource serves as a valuable tool for researchers interested in exploring different language model sizes' efficacy in text classification tasks. Overall, this study underscores considering resource-efficient small language models as viable solutions for specific data classification challenges by demonstrating their comparable performance to larger ones. It contributes valuable insights to the ongoing discourse on optimizing model size for text classification tasks.