# TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise

**Authors:** Nan He, Hanyu Lai, Chenyang Zhao, Zirui Cheng, Junting Pan, Ruoyu Qin, Ruofan Lu, Rui Lu, Yunchen Zhang, Gangming Zhao, Zhaohui Hou, Zhiyuan Huang, Shaoqing Lu, Ding Liang, Mingjie Zhan  
**arXiv:** [2310.19019](https://arxiv.org/abs/2310.19019)  
**Date:** 2023-10-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.19019v1/

## Key points

- Large Language Models (LLMs) in Natural Language Processing (NLP) have shown remarkable reasoning and data augmentation capabilities.
- The focus has mostly been on large models, leaving a gap in understanding the potential of smaller models.
- The authors propose a new model called TeacherLM-7.1B to address this gap.
- TeacherLM-7.1B is designed to annotate relevant fundamentals, chain of thought, and common mistakes for most NLP samples.
- It goes beyond providing just an answer and enables other models to learn not only "what" but also "why".
- TeacherLM-7.1B achieves a zero-shot score of 52.3 on MMLU, surpassing most models with over 100 billion parameters.
- TeacherLM-7.1B has data augmentation ability and can be used as a base for training various student models with different parameters from OPT and BLOOM series in a multi-task setting.
- Experimental results demonstrate that the data augmentation provided by TeacherLM brings significant benefits.
- The authors plan to release the TeacherLM series of models and augmented datasets as open-source resources.
- TeacherLM-7.1B is introduced as a powerful tool for teaching NLP models to understand not just the answers but also the underlying reasoning behind them.

## Summary

In the field of Natural Language Processing (NLP), Large Language Models (LLMs) have shown remarkable reasoning and data augmentation capabilities. However, the focus has mostly been on these large models, leaving a gap in understanding the potential of smaller models. To address this, the authors propose a new model called TeacherLM-7.1B. TeacherLM-7.1B is designed to annotate relevant fundamentals, chain of thought, and common mistakes for most NLP samples. This goes beyond providing just an answer and enables other models to learn not only "what" but also "why". The performance of TeacherLM-7.1B is impressive, achieving a zero-shot score of 52.3 on MMLU (a benchmark metric), surpassing most models with over 100 billion parameters. One notable feature of TeacherLM-7.1B is its data augmentation ability. Using this model as a base, the researchers augmented 58 NLP datasets and trained various student models with different parameters from OPT and BLOOM series in a multi-task setting. The experimental results demonstrate that the data augmentation provided by TeacherLM brings significant benefits. The authors plan to release the TeacherLM series of models and augmented datasets as open-source resources, allowing other researchers and developers to benefit from their work. Overall, this study introduces TeacherLM-7.1B as a powerful tool for teaching NLP models to understand not just the answers but also the underlying reasoning behind them. The model's impressive performance and data augmentation capabilities make it a valuable asset for advancing research in NLP tasks.