# Students' Perceptions and Preferences of Generative Artificial Intelligence Feedback for Programming

**Authors:** Zhengdong Zhang, Zihan Dong, Yang Shi, Noboru Matsuda, Thomas Price, Dongkuan Xu  
**arXiv:** [2312.11567](https://arxiv.org/abs/2312.11567)  
**Date:** 2023-12-17  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2312.11567v1/

## Key points

- Study by Zhang et al. (2023) focuses on using large language models (LLMs), specifically ChatGPT, for automating feedback in educational settings
- How do students perceive LLM-generated feedback in terms of its formative nature?
- What are the advantages of feedback prompts that include students' code versus those that do not?
- What suggestions do students have for enhancing AI-generated feedback?
- Automated feedback generated using ChatGPT API for four lab assignments in a CS1 class
- Students viewed AI-generated feedback as aligning well with established formative feedback guidelines
- Preference among students for feedback that incorporated their own code into the LLM prompt due to specificity, clarity, and corrective nature
- Students generally expected specific and corrective feedback with ample code examples but had varying opinions on the tone of the feedback provided by ChatGPT
- Findings demonstrate ChatGPT's effectiveness in generating Java programming assignment feedback perceived as formative by students
- Research highlights potential of AI-driven tools like ChatGPT in educational settings and identifies improvements needed to enhance AI-generated feedback for student learning outcomes

## Summary

The study conducted by Zhang et al. (2023) explores the use of artificial intelligence, specifically large language models (LLMs), for automating feedback in educational settings. The researchers focused on ChatGPT, a popular LLM, to provide feedback for Java programming assignments in a CS1 class. Their research aimed to address three key questions: 1) How do students perceive LLM-generated feedback in terms of its formative nature? 2) What are the comparative advantages of feedback prompts that include students' code versus those that do not? 3) What suggestions do students have for enhancing AI-generated feedback? To answer these questions, automated feedback was generated using the ChatGPT API for four lab assignments within the CS1 class. The results from surveys indicated that students viewed the AI-generated feedback as aligning well with established formative feedback guidelines. There was also a clear preference among students for feedback that incorporated their own code into the LLM prompt due to its specificity, clarity, and corrective nature. Additionally, the study revealed that students generally expected specific and corrective feedback with ample code examples but had varying opinions on the tone of the feedback provided by ChatGPT. Overall, the findings demonstrated that ChatGPT could effectively generate Java programming assignment feedback perceived as formative by students. This research not only showcases the potential of AI-driven tools like ChatGPT in educational settings but also sheds light on specific improvements necessary to enhance the utility of AI-generated feedback for student learning outcomes. By exploring students' perceptions and preferences regarding generative artificial intelligence feedback for programming assignments, this study contributes valuable insights to further optimize AI applications in educational contexts.