# Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models

**Authors:** Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen, Heng-Tze Cheng, Ed H. Chi, Quoc V Le, Denny Zhou  
**arXiv:** [2310.06117](https://arxiv.org/abs/2310.06117)  
**Date:** 2023-10-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.06117v1/

## Key points

- Technique called Step-Back Prompting aims to improve complex multi-step reasoning abilities of large language models (LLMs)
- Grounds reasoning on abstractions to reduce errors in intermediate reasoning steps
- Derives high-level concepts and first principles from instances containing specific details to guide the reasoning process
- Significant performance gains observed on STEM, Knowledge QA, and Multi-Hop Reasoning tasks using PaLM-2L models
- Improvements seen in MMLU Physics and Chemistry (7% and 11%), TimeQA (27%), and MuSiQue (7%)
- Builds upon previous research on process supervision with step-by-step verification and Chain-of-Thought prompting
- Inspired by human problem-solving strategies of stepping back and abstracting high-level concepts and principles
- Simple yet effective technique for improving complex multi-step reasoning in LLMs
- Experimental results demonstrate substantial performance gains across various challenging tasks

## Summary

In the paper titled "Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models," the authors present a technique called Step-Back Prompting that aims to improve the complex multi-step reasoning abilities of large language models (LLMs). They propose grounding reasoning on abstractions to reduce errors in intermediate reasoning steps. The technique involves deriving high-level concepts and first principles from instances containing specific details, which are then used to guide the reasoning process. The authors conducted experiments using PaLM-2L models and observed significant performance gains on various challenging reasoning-intensive tasks, including STEM, Knowledge QA, and Multi-Hop Reasoning. For example, Step-Back Prompting improved PaLM-2L performance on MMLU Physics and Chemistry by 7% and 11%, TimeQA by 27%, and MuSiQue by 7%. The paper builds upon previous research that showed process supervision with step-by-step verification as a promising approach to improve intermediate reasoning steps' correctness. Techniques like Chain-of-Thought prompting were introduced to produce coherent series of intermediate reasoning steps. However, inspired by human problem solving strategies where individuals often step back and abstract high level concepts and principles to guide their thinking process, the authors propose Step-Back Prompting as an additional method to enhance LLMs' reasoning capabilities. Overall, this paper introduces a simple yet effective technique for improving complex multi-step reasoning in LLMs. The experimental results demonstrate substantial performance gains across various challenging tasks, highlighting the potential of Step-Back Prompting in enhancing LLMs' abilities to follow correct reasoning paths towards solutions.