Results of the summarizing process for the arXiv paper: 2409.17539v1

, , , , In recent years, Large Language Models (LLMs) have shown impressive capabilities in various NLP tasks. However, they still struggle with complex logical reasoning tasks. To enhance their logical reasoning abilities, Chain-of-Thought (CoT) prompting has been introduced as a method to add intermediate steps in the reasoning process. While this has shown promise, existing methods like CoT suffer from unfaithful reasoning where derived conclusions may not align with the generated chain of thought. To address this challenge, some studies have integrated propositional logic to improve LLMs' logical reasoning abilities. However, there are potential omissions in extracting logical expressions that can lead to information loss and incorrect results. To overcome these limitations, a new approach called Logic-of-Thought (LoT) prompting is proposed in this paper. LoT utilizes propositional logic to extract expanded logical information from the input context and integrates it as additional augmentation to the input prompts for LLMs. The LoT approach aims to enhance the capability of logical reasoning while preserving the original input prompt. It is complementary to existing prompting methods and can be seamlessly integrated with them. Extensive experiments have demonstrated that LoT significantly boosts the performance of various prompting methods across five logical reasoning tasks. For instance, it enhances Chain-of-Thought's performance on the ReClor dataset by +4.35%, improves Chain-of-Thought with Self-Consistency on LogiQA by +5%, and boosts Tree-of-Thoughts on ProofWriter dataset by +8%. By incorporating expanded logical information into the input prompts, LoT effectively guides LLMs in producing more accurate and reliable results in logical reasoning tasks compared to traditional prompting methods.

• Large Language Models (LLMs) excel in NLP tasks but struggle with complex logical reasoning.
• Chain-of-Thought (CoT) prompting introduces intermediate steps for logical reasoning enhancement.
• Existing methods like CoT may suffer from unfaithful reasoning, leading to misaligned conclusions.
• Some studies integrate propositional logic to improve LLMs' logical reasoning abilities.
• Logic-of-Thought (LoT) prompting proposes using propositional logic to extract expanded logical information and augment input prompts for LLMs.
• LoT aims to enhance logical reasoning while preserving the original input prompt and can be seamlessly integrated with existing methods.
• Extensive experiments show that LoT significantly improves performance across various logical reasoning tasks:
• Enhances Chain-of-Thought's performance on ReClor dataset by +4.35%
• Improves Chain-of-Thought with Self-Consistency on LogiQA by +5%
• Boosts Tree-of-Thoughts on ProofWriter dataset by +8%
• LoT guides LLMs to produce more accurate and reliable results in logical reasoning tasks compared to traditional prompting methods.

Summary- Large Language Models (LLMs) are good at understanding language but struggle with complex thinking. - Chain-of-Thought (CoT) helps LLMs think better by adding steps in between. - Sometimes, methods like CoT can make mistakes in thinking and reach the wrong answers. - Logic-of-Thought (LoT) uses a type of logic to help LLMs think more clearly and improve their reasoning skills. - LoT makes LLMs smarter at thinking logically without changing the original question. Definitions- Large Language Models (LLMs): Advanced computer programs that understand and generate human language. - Logical Reasoning: Thinking carefully to solve problems or answer questions using clear steps and rules. - Propositional Logic: A type of formal logic that deals with statements as true or false based on logical rules.

Introduction In recent years, Large Language Models (LLMs) have shown impressive capabilities in various Natural Language Processing (NLP) tasks. However, they still struggle with complex logical reasoning tasks. To enhance their logical reasoning abilities, Chain-of-Thought (CoT) prompting has been introduced as a method to add intermediate steps in the reasoning process. While this has shown promise, existing methods like CoT suffer from unfaithful reasoning where derived conclusions may not align with the generated chain of thought. To address this challenge, some studies have integrated propositional logic to improve LLMs' logical reasoning abilities. However, there are potential omissions in extracting logical expressions that can lead to information loss and incorrect results. To overcome these limitations, a new approach called Logic-of-Thought (LoT) prompting is proposed in this paper. What is LoT Prompting? LoT utilizes propositional logic to extract expanded logical information from the input context and integrates it as additional augmentation to the input prompts for LLMs. The LoT approach aims to enhance the capability of logical reasoning while preserving the original input prompt. It is complementary to existing prompting methods and can be seamlessly integrated with them. How Does LoT Work? The first step of LoT prompting involves identifying relevant propositions from the input context using natural language processing techniques such as Named Entity Recognition (NER). These propositions are then converted into formal logic expressions using pre-defined rules and templates. Next, these expanded logic expressions are incorporated into the original input prompt through concatenation or insertion at specific positions within the prompt. This creates an augmented prompt that contains both natural language and formal logic expressions. Finally, this augmented prompt is fed into an LLM for inference on a given task. By incorporating expanded logical information into the input prompts, LoT effectively guides LLMs in producing more accurate and reliable results in logical reasoning tasks compared to traditional prompting methods. Experimental Results To evaluate the effectiveness of LoT prompting, extensive experiments were conducted on five logical reasoning tasks: ReClor, LogiQA, ProofWriter, CSQA, and ARCT. These tasks cover a range of logical reasoning abilities such as deduction, induction, and analogy. The results showed that LoT significantly boosts the performance of various prompting methods across all five tasks. For instance, it enhances Chain-of-Thought's performance on the ReClor dataset by +4.35%, improves Chain-of-Thought with Self-Consistency on LogiQA by +5%, and boosts Tree-of-Thoughts on ProofWriter dataset by +8%. Moreover, LoT also outperforms existing methods that integrate propositional logic into LLMs' input prompts. This is because LoT addresses potential omissions in extracting logical expressions from the input context and ensures that all relevant information is incorporated into the augmented prompt. Conclusion In conclusion, this research paper introduces Logic-of-Thought (LoT) prompting as a novel approach to enhance LLMs' logical reasoning abilities. By incorporating expanded logical information into the input prompts for LLMs, LoT effectively guides them in producing more accurate and reliable results in various logical reasoning tasks. The experimental results demonstrate that LoT significantly improves the performance of existing prompting methods across different tasks. It also outperforms other approaches that integrate propositional logic into LLMs' input prompts. Future work could explore further improvements to LoT by incorporating other types of logic or exploring different ways to incorporate formal logic expressions into input prompts. Overall, this research opens up new possibilities for enhancing LLMs' capabilities in complex logical reasoning tasks through innovative prompting techniques like Logic-of-Thought.