# Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost

**Authors:** Sania Nayab, Giulio Rossolini, Giorgio Buttazzo, Nicolamaria Manes, Fabrizio Giacomelli  
**arXiv:** [2407.19825](https://arxiv.org/abs/2407.19825)  
**Date:** 2024-07-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2407.19825v1/

## Key points

- Large language models (LLMs) have shown impressive capabilities in handling complex question-answering tasks, marking significant progress in natural language understanding and generative AI.
- Advancements in architectures and training methods have played a pivotal role in improving the performance of LLMs.
- Prompt engineering techniques, such as chain-of-thought (CoT), have evolved significantly to enhance explanation and correctness of outputs.
- A challenge faced by LLMs is the time required to generate answers with detailed reasoning, leading to lengthy outputs.
- A refined prompt engineering strategy called Constrained-CoT (CCoT) has been developed to address this issue by encouraging models to limit output length while maintaining accuracy.
- Experimental results demonstrate the benefits of CCoT across various models, showing improvements in accuracy and response times for large models while addressing limitations across different model sizes.

## Summary

In recent years, large language models (LLMs) have shown impressive capabilities in handling complex question-answering tasks. This marks significant progress in natural language understanding and generative AI. The advancements in architectures and training methods have played a pivotal role in improving the performance of these models. Prompt engineering techniques, such as chain-of-thought (CoT), have also evolved significantly to enhance the explanation and correctness of outputs. However, one challenge faced by these models is the time required to generate answers with detailed reasoning. This often leads to lengthy outputs. To address this issue, a refined prompt engineering strategy called Constrained-CoT (CCoT) has been developed. CCoT encourages models to limit their output length while maintaining accuracy. Experimental results on pre-trained LLMs demonstrate the benefits of the proposed metrics and the efficacy of CCoT across various models. For instance, constraining the reasoning of LLaMA2-70b to 100 words using CCoT improves accuracy from 36.01% (with CoT) to 41.07% on the GSM8K dataset while reducing average output length by 28 words. This work highlights the importance of concise reasoning for question-answering tasks and offers valuable insights into leveraging CoT effectively and guiding future LLM training practices. It makes significant contributions by proposing new metrics for evaluating correctness while considering conciseness, introducing CCoT as a prompt engineering strategy for enhancing time-predictability in LLMs, and presenting experimental results that showcase improvements in accuracy and response times for large models while addressing limitations across different model sizes. The paper is structured as follows: Section 2 reviews related literature; Section 3 provides motivation for the study; Section 4 introduces metrics focusing on conciseness; Section 5 presents CCoT approach; Section 6 discusses experimental results on diverse pre-trained models; and finally, Section 7 concludes and suggests future research directions.