# Orca 2: Teaching Small Language Models How to Reason

**Authors:** Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, Hamid Palangi, Guoqing Zheng, Corby Rosset, Hamed Khanpour, Ahmed Awadallah  
**arXiv:** [2311.11045](https://arxiv.org/abs/2311.11045)  
**Date:** 2023-11-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2311.11045v2/

## Key points

- Orca 1 has demonstrated superior performance on benchmarks like BigBench Hard and AGIEval by learning from rich signals such as explanation traces.
- Orca 2 aims to enhance the reasoning abilities of smaller language models (LMs) by exploring improved training signals.
- Previous research on training small LMs has relied heavily on imitation learning, but Orca 2 believes excessive emphasis on imitation may limit the potential of smaller models.
- Orca 2 focuses on teaching small LMs different solution strategies for various tasks, which may differ from those employed by larger models.
- Orca 2 trains the model in various reasoning techniques such as step-by-step analysis, recall then generate, recall-reason-generate, and direct answer approaches.
- Orca 2 surpasses models of similar size and achieves comparable or better performance levels compared to models that are 5-10 times larger.
- Evaluations specifically focus on complex tasks that test advanced reasoning abilities in zero-shot settings.
- Safety evaluation is an important aspect considered in this study, with experiments conducted using publicly available datasets related to implicit and explicit toxicity, truthfulness, content harms across different domains (IP), and jailbreaks.
- Two evaluation regimes are employed: discriminative evaluation where the model classifies given content types accurately and generative evaluation where the model produces output that adheres to safety guidelines.
- Certain models perform better at classifying toxic statements than neutral statements, raising concerns about potential erasure of content related to specific identity groups even if it is not problematic. However, Orca 2 family models do not exhibit this problem.
- Instruction following rates for various models are assessed with high rates observed for most models except for Orca 1.
- System instructions influence GPT-4's response and its ability to engage in careful thinking. The strategy employed by an LM can significantly affect its performance when reasoning about a task.

## Summary

Orca 1 has demonstrated superior performance on benchmarks like BigBench Hard and AGIEval by learning from rich signals such as explanation traces. Building on this success, Orca 2 aims to enhance the reasoning abilities of smaller language models (LMs) by exploring improved training signals. Previous research on training small LMs has relied heavily on imitation learning, replicating the output of larger models. However, Orca 2 believes that excessive emphasis on imitation may limit the potential of smaller models. In Orca 2, the focus is on teaching small LMs different solution strategies for various tasks, which may differ from those employed by larger models. While larger models might directly answer complex tasks, smaller models may not have the same capacity. Therefore, Orca 2 trains the model in various reasoning techniques such as step-by-step analysis, recall then generate, recall-reason-generate, and direct answer approaches. The goal is to enable the model to determine the most effective solution strategy for each task. To evaluate Orca 2's performance, a comprehensive set of 15 diverse benchmarks consisting of approximately 100 tasks and over 36,000 unique prompts are used. Remarkably, Orca 2 surpasses models of similar size and achieves comparable or better performance levels compared to models that are 5-10 times larger. These evaluations specifically focus on complex tasks that test advanced reasoning abilities in zero-shot settings. Additionally, safety evaluation is an important aspect considered in this study. Experiments are conducted using publicly available datasets related to implicit and explicit toxicity, truthfulness, content harms across different domains (IP), and jailbreaks. Two evaluation regimes are employed: discriminative evaluation where the model classifies given content types accurately and generative evaluation where the model produces output that adheres to safety guidelines. The experiments reveal that certain models perform better at classifying toxic statements than neutral statements. This observation raises concerns about potential erasure of content related to specific identity groups even if it is not problematic. However, models in the Orca 2 family, LLaMa-2 family and WizardLM family do not exhibit this problem. Instruction following rates for various models are also assessed with high rates observed for most models except for Orca 1. Overall these findings highlight the influence of system instructions on GPT-4's response and its ability to engage in careful thinking as well as demonstrate how strategy employed by an LM can affect its performance significantly when reasoning about a task . By expanding research on smaller LMs and providing publicly available weights for Orca 2 at aka ms/orca lm , this study aims to support further development , evaluation , alignment ,and potentially improvement of their capabilities even more .