# ChatUniTest: a ChatGPT-based automated unit test generation tool

**Authors:** Zhuokui Xie, Yinghao Chen, Chen Zhi, Shuiguang Deng, Jianwei Yin  
**arXiv:** [2305.04764](https://arxiv.org/abs/2305.04764)  
**Date:** 2023-05-08  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2305.04764v1/

## Key points

- Unit testing is essential but can be tedious and time-consuming for developers.
- Existing automated unit test generation tools like EvoSuite and Randoop lack program comprehension, resulting in tests with poor readability and limited assertions.
- Language-model-based tools like AthenaTest and A3Test also have limitations in generating correct unit tests.
- The authors introduce ChatUniTest, a ChatGPT-based automated unit test generation tool developed under the Generation-Validation-Repair framework.
- ChatUniTest works by parsing the project to extract crucial information and creating an adaptive focal context that includes the focal method and its dependencies within a predefined maximum prompt token limit.
- This context is incorporated into a prompt submitted to ChatGPT for generating a response that is used to extract the raw test.
- ChatUniTest validates generated tests and employs rule-based repair techniques for fixing syntactic and simple compile errors, while using ChatGPT-based repair for more challenging errors.
- An evaluation conducted by the authors demonstrates that ChatUniTest surpasses EvoSuite in terms of branch and line coverage.
- It also outperforms AthenaTest and A3Test in focal method coverage while effectively generating assertions using mock objects and reflection to achieve test objectives.

## Summary

Unit testing is an essential task for developers that can be tedious and time-consuming. To address this issue, automated unit test generation tools have been developed such as EvoSuite and Randoop. However, these existing tools lack program comprehension resulting in tests with poor readability and limited assertions. Language-model-based tools like AthenaTest and A3Test also have limitations in generating correct unit tests. In this paper, the authors introduce ChatUniTest - a ChatGPT-based automated unit test generation tool developed under the Generation-Validation-Repair framework. It works by parsing the project to extract crucial information and creating an adaptive focal context that includes the focal method and its dependencies within a predefined maximum prompt token limit. This context is incorporated into a prompt which is submitted to ChatGPT for generating a response that is used to extract the raw test. To ensure quality of generated tests, ChatUniTest validates them and employs rule-based repair techniques to fix syntactic and simple compile errors while using ChatGPT-based repair for more challenging errors. The authors conducted an evaluation demonstrating superiority of ChatUniTest over EvoSuite in terms of branch and line coverage as well as surpassing AthenaTest and A3Test in focal method coverage while effectively generating assertions using mock objects and reflection to achieve test objectives.