In November 2022, OpenAI introduced ChatGPT, a chatbot based on supervised and reinforcement learning that can answer questions with human-like responses and generate code from scratch or complete coding templates. The release of ChatGPT sparked a debate about its use in academia, particularly by students. Surprisingly, a survey conducted at POLITEHNICA University of Bucharest (UPB) revealed that over 40% of students had been using generative AI tools like ChatGPT for at least six months for coding assignments. Further exploration into the tool's capabilities showed that ChatGPT provided correct answers in 68% of cases but only around half were deemed legible solutions beneficial to students. However, ChatGPT excelled in performing code reviews on student programming homework. A study on 72 programming tasks from the Functional Programming course at UPB found that ChatGPT scored approximately 7 out of 10 for correctness but struggled with efficiency and comprehensibility. Additionally, the evaluation highlighted that ChatGPT could improve its score to 8.6 when errors were pointed out, although this required a certain level of expertise from students. Generating tests for solutions also proved challenging as only 70% were accurate, leading to potential confusion for students when tests failed. The survey results indicated that generative AI tools like ChatGPT are commonly used by students for homework and exams. To gain a better understanding of their accuracy and practicality in education, further assessment is needed to determine how these tools can provide correct and useful results for students. The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience. Overall, while ChatGPT shows promise in assisting with coding assignments and providing feedback on student work, there are limitations regarding efficiency and comprehension that need to be addressed for optimal educational use. Further research is necessary to explore ways to effectively integrate generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes for students.
Summary- OpenAI introduced ChatGPT in November 2022, a tool that can answer questions and generate code.
- ChatGPT learns through supervised and reinforcement learning methods to provide human-like responses.
- A survey at POLITEHNICA University found that many students use ChatGPT for coding assignments.
- While ChatGPT is good at reviewing code, it struggles with efficiency and clarity in some cases.
- More research is needed to understand how generative AI tools like ChatGPT can best help students with their programming tasks.
Definitions
- OpenAI: A company that develops artificial intelligence technology.
- Supervised learning: A method where the AI learns from labeled data provided by humans.
- Reinforcement learning: A method where the AI learns through trial and error based on rewards or punishments.
- Generative AI: Artificial intelligence that can create new content such as text or code.
In November 2022, OpenAI made a groundbreaking announcement - the release of ChatGPT, a chatbot based on supervised and reinforcement learning. This chatbot was designed to answer questions with human-like responses and even generate code from scratch or complete coding templates. The introduction of ChatGPT sparked a debate about its use in academia, particularly by students.
The potential impact of this technology on education is immense. With the ability to provide correct answers and assist with coding assignments, students may be tempted to rely heavily on ChatGPT for their coursework. But how effective is this tool really? A survey conducted at POLITEHNICA University of Bucharest (UPB) revealed some interesting insights into the use of generative AI tools like ChatGPT among students.
According to the survey results, over 40% of students had been using generative AI tools for at least six months for coding assignments. This indicates that these tools are already widely used by students in their academic pursuits. However, further exploration into ChatGPT's capabilities showed that while it provided correct answers in 68% of cases, only around half were deemed legible solutions beneficial to students.
One area where ChatGPT excelled was performing code reviews on student programming homework. A study conducted on 72 programming tasks from the Functional Programming course at UPB found that ChatGPT scored approximately 7 out of 10 for correctness but struggled with efficiency and comprehensibility. This highlights one potential benefit of using generative AI tools - they can provide valuable feedback on student work.
However, there were also limitations identified in the evaluation process. For instance, when errors were pointed out by experts or instructors, ChatGPT's score improved significantly to an average of 8.6 out of 10. This suggests that while the tool has potential, it may require a certain level of expertise from students to achieve optimal results.
Another challenge identified in the evaluation was ChatGPT's ability to generate tests for solutions. The study found that only 70% of the generated tests were accurate, which could potentially lead to confusion for students when their code fails these tests. This highlights the need for further research and development to improve the accuracy and usefulness of generative AI tools like ChatGPT in an educational setting.
To gain a better understanding of their accuracy and practicality in education, further assessment is needed to determine how these tools can provide correct and useful results for students. The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience. This provides a solid foundation for future studies to build upon.
Overall, while ChatGPT shows promise in assisting with coding assignments and providing feedback on student work, there are limitations regarding efficiency and comprehension that need to be addressed for optimal educational use. Further research is necessary to explore ways to effectively integrate generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes for students.
In conclusion, the release of ChatGPT has sparked a debate about its use in academia, particularly by students. While it has shown potential in assisting with coding assignments and providing feedback on student work, there are still challenges that need to be addressed before it can be fully integrated into education as a valuable tool. With further research and development, generative AI tools like ChatGPT have the potential to revolutionize how we learn programming skills.
In November 2022, OpenAI has introduced ChatGPT, a chatbot based on supervised and reinforcement learning. Not only can it answer questions emulating human-like responses, but it can also generate code from scratch or complete coding templates provided by the user. ChatGPT can generate unique responses which render any traditional anti-plagiarism tool useless. Its release has ignited a heated debate about its usage in academia, especially by students. We have found, to our surprise, that our students at POLITEHNICA University of Bucharest (UPB) have been using generative AI tools (ChatGPT and its predecessors) for solving homework, for at least 6 months. We therefore set out to explore the capabilities of ChatGPT and assess its value for educational purposes. We solved all our coding assignments for the semester from our UPB Functional Programming course. We discovered that, although ChatGPT provides correct answers in 68% of the cases, only around half of those are legible solutions which can benefit students in some form. On the other hand, ChatGPT has a very good ability to perform code review on student programming homework. Based on these findings, we discuss the pros and cons of ChatGPT in education.
