# Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilities

**Authors:** Yu Nong, Mohammed Aldeen, Long Cheng, Hongxin Hu, Feng Chen, Haipeng Cai  
**arXiv:** [2402.17230](https://arxiv.org/abs/2402.17230)  
**Date:** 2024-02-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.17230v1/

## Key points

- Vulnerabilities in software security are a significant threat to modern society, with their prevalence increasing in recent years.
- Deep learning (DL) approaches have gained traction for addressing vulnerabilities due to their ability to overcome traditional barriers.
- Large language models (LLMs) and chain-of-thought (CoT) prompting have emerged as promising solutions for software vulnerability analysis tasks.
- The study introduces VSP, a vulnerability-semantics-guided prompting approach that demonstrates superior performance compared to five baselines on three LLMs and two datasets.
- Challenges faced by LLMs in analyzing vulnerabilities include insufficient context leading to failures in real-world samples and 'forgetting' the Common Weakness Enumeration (CWE) due to lengthy input text.
- Suggestions for improvement include providing additional context information in the code through comments, relocating questions after code samples, explicitly defining CWE meanings, and using supplementary methods for thorough analysis of complex real-world code structures.

## Summary

, , , , 

In the realm of software security, vulnerabilities pose a significant threat to modern society, with their prevalence increasing in recent years. While various defense mechanisms have been proposed, deep learning (DL) approaches have gained traction due to their ability to overcome traditional barriers. However, DL-based methods face challenges such as limited datasets and difficulty in generalizing to real-world scenarios. <break>

<break>

To address these challenges, large language models (LLMs) have emerged as a promising solution, particularly through chain-of-thought (CoT) prompting. This paper delves into leveraging LLMs and CoT for three key software vulnerability analysis tasks: identifying vulnerabilities, discovering new vulnerabilities, and patching detected vulnerabilities. The authors introduce VSP, a vulnerability-semantics-guided prompting approach that instantiates the CoT methodology for these tasks. Extensive experiments comparing VSP against five baselines on three LLMs and two datasets demonstrate its superior performance across all tasks. The study also uncovers challenges faced by LLMs in analyzing vulnerabilities, such as insufficient context leading to failures in real-world samples. To address this issue, the authors suggest providing additional context information in the code through comments. Furthermore, they highlight instances where LLMs 'forget' the Common Weakness Enumeration (CWE) they are analyzing due to lengthy input text. A potential solution proposed is relocating questions after code samples and explicitly defining CWE meanings. Additionally, the paper discusses how LLMs fall short in conducting comprehensive control and data flow analysis compared to conventional techniques. The complexity of real-world code structures poses a challenge for LLMs, necessitating supplementary methods for thorough analysis. Overall, this study showcases the effectiveness of CoT-inspired prompting using LLMs for software vulnerability analysis tasks while shedding light on areas for improvement in leveraging these models effectively.