ChatGPT in the classroom. Exploring its potential and limitations in a Functional Programming course

AI-generated keywords: OpenAI ChatGPT generative AI tools coding assignments education

AI-generated Key Points

OpenAI introduced ChatGPT in November 2022
ChatGPT is based on supervised and reinforcement learning
ChatGPT can answer questions with human-like responses and generate code from scratch or complete coding templates
A survey at POLITEHNICA University of Bucharest (UPB) found over 40% of students had been using generative AI tools like ChatGPT for at least six months for coding assignments
ChatGPT provided correct answers in 68% of cases, but only around half were deemed legible solutions beneficial to students
ChatGPT excelled in performing code reviews on student programming homework
ChatGPT scored approximately 7 out of 10 for correctness in a study on 72 programming tasks from the Functional Programming course at UPB, but struggled with efficiency and comprehensibility
The evaluation highlighted that errors pointed out by students could improve ChatGPT's score to 8.6, requiring a certain level of expertise from students
Generating tests for solutions proved challenging as only 70% were accurate, potentially causing confusion when tests failed for students
Generative AI tools like ChatGPT are commonly used by students for homework and exams according to survey results
Further assessment is needed to determine the accuracy and practicality of these tools in education, especially regarding providing correct and useful results for students
The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience
While promising in assisting with coding assignments and providing feedback on student work, limitations exist regarding efficiency and comprehension that need addressing for optimal educational use
Further research is necessary to explore effective integration of generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes

Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Dan-Matei Popovici

International Journal of Human-Computer Interaction, 2023

arXiv: 2401.11166v1 - DOI (cs.HC)

License: CC BY 4.0

Abstract: In November 2022, OpenAI has introduced ChatGPT, a chatbot based on supervised and reinforcement learning. Not only can it answer questions emulating human-like responses, but it can also generate code from scratch or complete coding templates provided by the user. ChatGPT can generate unique responses which render any traditional anti-plagiarism tool useless. Its release has ignited a heated debate about its usage in academia, especially by students. We have found, to our surprise, that our students at POLITEHNICA University of Bucharest (UPB) have been using generative AI tools (ChatGPT and its predecessors) for solving homework, for at least 6 months. We therefore set out to explore the capabilities of ChatGPT and assess its value for educational purposes. We solved all our coding assignments for the semester from our UPB Functional Programming course. We discovered that, although ChatGPT provides correct answers in 68% of the cases, only around half of those are legible solutions which can benefit students in some form. On the other hand, ChatGPT has a very good ability to perform code review on student programming homework. Based on these findings, we discuss the pros and cons of ChatGPT in education.

Submitted to arXiv on 20 Jan. 2024

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2401.11166v1

Comprehensive Summary
Key points
Layman's Summary
Blog article

In November 2022, OpenAI introduced ChatGPT, a chatbot based on supervised and reinforcement learning that can answer questions with human-like responses and generate code from scratch or complete coding templates. The release of ChatGPT sparked a debate about its use in academia, particularly by students. Surprisingly, a survey conducted at POLITEHNICA University of Bucharest (UPB) revealed that over 40% of students had been using generative AI tools like ChatGPT for at least six months for coding assignments. Further exploration into the tool's capabilities showed that ChatGPT provided correct answers in 68% of cases but only around half were deemed legible solutions beneficial to students. However, ChatGPT excelled in performing code reviews on student programming homework. A study on 72 programming tasks from the Functional Programming course at UPB found that ChatGPT scored approximately 7 out of 10 for correctness but struggled with efficiency and comprehensibility. Additionally, the evaluation highlighted that ChatGPT could improve its score to 8.6 when errors were pointed out, although this required a certain level of expertise from students. Generating tests for solutions also proved challenging as only 70% were accurate, leading to potential confusion for students when tests failed. The survey results indicated that generative AI tools like ChatGPT are commonly used by students for homework and exams. To gain a better understanding of their accuracy and practicality in education, further assessment is needed to determine how these tools can provide correct and useful results for students. The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience. Overall, while ChatGPT shows promise in assisting with coding assignments and providing feedback on student work, there are limitations regarding efficiency and comprehension that need to be addressed for optimal educational use. Further research is necessary to explore ways to effectively integrate generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes for students.

- OpenAI introduced ChatGPT in November 2022
- ChatGPT is based on supervised and reinforcement learning
- ChatGPT can answer questions with human-like responses and generate code from scratch or complete coding templates
- A survey at POLITEHNICA University of Bucharest (UPB) found over 40% of students had been using generative AI tools like ChatGPT for at least six months for coding assignments
- ChatGPT provided correct answers in 68% of cases, but only around half were deemed legible solutions beneficial to students
- ChatGPT excelled in performing code reviews on student programming homework
- ChatGPT scored approximately 7 out of 10 for correctness in a study on 72 programming tasks from the Functional Programming course at UPB, but struggled with efficiency and comprehensibility
- The evaluation highlighted that errors pointed out by students could improve ChatGPT's score to 8.6, requiring a certain level of expertise from students
- Generating tests for solutions proved challenging as only 70% were accurate, potentially causing confusion when tests failed for students
- Generative AI tools like ChatGPT are commonly used by students for homework and exams according to survey results
- Further assessment is needed to determine the accuracy and practicality of these tools in education, especially regarding providing correct and useful results for students
- The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience
- While promising in assisting with coding assignments and providing feedback on student work, limitations exist regarding efficiency and comprehension that need addressing for optimal educational use
- Further research is necessary to explore effective integration of generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes

Summary- OpenAI introduced ChatGPT in November 2022, a tool that can answer questions and generate code. - ChatGPT learns through supervised and reinforcement learning methods to provide human-like responses. - A survey at POLITEHNICA University found that many students use ChatGPT for coding assignments. - While ChatGPT is good at reviewing code, it struggles with efficiency and clarity in some cases. - More research is needed to understand how generative AI tools like ChatGPT can best help students with their programming tasks. Definitions- OpenAI: A company that develops artificial intelligence technology. - Supervised learning: A method where the AI learns from labeled data provided by humans. - Reinforcement learning: A method where the AI learns through trial and error based on rewards or punishments. - Generative AI: Artificial intelligence that can create new content such as text or code.

In November 2022, OpenAI made a groundbreaking announcement - the release of ChatGPT, a chatbot based on supervised and reinforcement learning. This chatbot was designed to answer questions with human-like responses and even generate code from scratch or complete coding templates. The introduction of ChatGPT sparked a debate about its use in academia, particularly by students. The potential impact of this technology on education is immense. With the ability to provide correct answers and assist with coding assignments, students may be tempted to rely heavily on ChatGPT for their coursework. But how effective is this tool really? A survey conducted at POLITEHNICA University of Bucharest (UPB) revealed some interesting insights into the use of generative AI tools like ChatGPT among students. According to the survey results, over 40% of students had been using generative AI tools for at least six months for coding assignments. This indicates that these tools are already widely used by students in their academic pursuits. However, further exploration into ChatGPT's capabilities showed that while it provided correct answers in 68% of cases, only around half were deemed legible solutions beneficial to students. One area where ChatGPT excelled was performing code reviews on student programming homework. A study conducted on 72 programming tasks from the Functional Programming course at UPB found that ChatGPT scored approximately 7 out of 10 for correctness but struggled with efficiency and comprehensibility. This highlights one potential benefit of using generative AI tools - they can provide valuable feedback on student work. However, there were also limitations identified in the evaluation process. For instance, when errors were pointed out by experts or instructors, ChatGPT's score improved significantly to an average of 8.6 out of 10. This suggests that while the tool has potential, it may require a certain level of expertise from students to achieve optimal results. Another challenge identified in the evaluation was ChatGPT's ability to generate tests for solutions. The study found that only 70% of the generated tests were accurate, which could potentially lead to confusion for students when their code fails these tests. This highlights the need for further research and development to improve the accuracy and usefulness of generative AI tools like ChatGPT in an educational setting. To gain a better understanding of their accuracy and practicality in education, further assessment is needed to determine how these tools can provide correct and useful results for students. The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience. This provides a solid foundation for future studies to build upon. Overall, while ChatGPT shows promise in assisting with coding assignments and providing feedback on student work, there are limitations regarding efficiency and comprehension that need to be addressed for optimal educational use. Further research is necessary to explore ways to effectively integrate generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes for students. In conclusion, the release of ChatGPT has sparked a debate about its use in academia, particularly by students. While it has shown potential in assisting with coding assignments and providing feedback on student work, there are still challenges that need to be addressed before it can be fully integrated into education as a valuable tool. With further research and development, generative AI tools like ChatGPT have the potential to revolutionize how we learn programming skills.

Created on 19 Mar. 2024

Assess the quality of the AI-generated content by voting

Score: 0

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.