Results of the summarizing process for the arXiv paper: 2405.00602v1

In the realm of automatic grading and feedback systems, there has been a longstanding exploration of traditional machine learning and deep learning techniques utilizing language models. The recent availability of high-performing large language models (LLMs) like LLaMA-2 has opened up avenues for investigating their application in automatic grading and feedback generation. However, these LLMs demand significant computational resources for fine-tuning and specific adjustments to optimize their performance for such tasks. To address these challenges, Parameter Efficient Fine-tuning (PEFT) methods such as LoRA and QLoRA have been implemented to reduce memory and computational requirements during model fine-tuning. This study delves into the effectiveness of PEFT-based quantized models with classification or regression heads in fine-tuning LLMs for automatically assigning continuous numerical grades to short answers and essays, as well as generating corresponding feedback. Experiments were conducted on both proprietary and open-source datasets, revealing that grade score predictions through finetuned LLMs are highly accurate with an average error rate of less than 3% in grade percentage. Moreover, when providing graded feedback, fine-tuned 4-bit quantized LLaMA-2 13B models outperformed competitive base models by achieving high similarity with subject matter expert feedback based on BLEU and ROUGE scores. Qualitatively, the feedback generated by these models also demonstrated a high level of alignment with expert feedback. The findings from this study shed light on the potential impact of using quantization approaches to fine-tune LLMs for various downstream tasks like automatic short answer scoring and feedback generation at reduced costs and latency. With advancements in AI-powered grading systems leveraging natural language processing technologies, educators can benefit from time-saving advantages while offering personalized feedback to students in settings where instructor resources may be limited, such as online education environments.

• - Automatic grading and feedback systems have traditionally utilized machine learning and deep learning techniques, with recent focus on large language models (LLMs) like LLaMA-2.
• - LLMs require significant computational resources for fine-tuning, leading to the development of Parameter Efficient Fine-tuning (PEFT) methods such as LoRA and QLoRA to reduce memory and computational requirements.
• - PEFT-based quantized models have been effective in fine-tuning LLMs for assigning grades to short answers and essays, as well as generating feedback with high accuracy.
• - Experiments showed that 4-bit quantized LLaMA-2 13B models outperformed base models in grade score predictions and feedback generation, achieving high similarity with expert feedback based on BLEU and ROUGE scores.
• - The study highlights the potential impact of using quantization approaches to fine-tune LLMs for automatic grading and feedback generation tasks at reduced costs and latency, benefiting educators in online education environments.

Summary1. Machines that can grade and give feedback automatically have been using advanced techniques like machine learning and deep learning, focusing on large language models. 2. These large language models need a lot of computer power to work well, so new methods like PEFT have been developed to make them more efficient. 3. By using these efficient methods, we can improve the way machines grade short answers and essays, providing accurate feedback. 4. Tests have shown that using quantized models with fewer bits can actually perform better than the original models in predicting grades and generating feedback. 5. Using these techniques can help teachers save time and money when grading assignments online. Definitions- Automatic grading: When a machine checks your work and gives you a score without needing a human to do it. - Feedback: Information or advice given to help improve something. - Machine learning: A type of technology that allows computers to learn from data and make decisions without being explicitly programmed. - Computational resources: The tools and power needed for computers to process information effectively. - Quantized models: Models that use less detailed information by reducing the number of bits used for calculations.

In recent years, there has been a growing interest in the use of automatic grading and feedback systems in education. These systems aim to provide efficient and accurate assessment of student work, as well as personalized feedback to aid in their learning process. Traditional machine learning and deep learning techniques have been extensively explored for this purpose, but with the availability of high-performing large language models (LLMs) like LLaMA-2, new avenues for research have opened up. However, utilizing these LLMs for automatic grading and feedback generation comes with its own set of challenges. These models require significant computational resources for fine-tuning and specific adjustments to optimize their performance for such tasks. To address these challenges, Parameter Efficient Fine-tuning (PEFT) methods such as LoRA and QLoRA have been developed. The study at hand delves into the effectiveness of PEFT-based quantized models in fine-tuning LLMs for automatically assigning continuous numerical grades to short answers and essays, as well as generating corresponding feedback. The experiments were conducted on both proprietary and open-source datasets, revealing promising results. One key finding from the study is that grade score predictions through finetuned LLMs are highly accurate with an average error rate of less than 3% in grade percentage. This demonstrates the potential of using LLMs for automatic grading tasks. Moreover, when it comes to providing graded feedback, fine-tuned 4-bit quantized LLaMA-2 13B models outperformed competitive base models by achieving high similarity with subject matter expert feedback based on BLEU and ROUGE scores. This indicates that these models are not only effective at predicting grades but also at generating relevant feedback that aligns with expert opinions. Qualitatively, the generated feedback by these models also demonstrated a high level of alignment with expert feedback. This further strengthens the argument for using quantization approaches to fine-tune LLMs for automatic grading and feedback tasks. The potential impact of these findings is significant. With advancements in AI-powered grading systems leveraging natural language processing technologies, educators can benefit from time-saving advantages while offering personalized feedback to students. This is especially beneficial in settings where instructor resources may be limited, such as online education environments. In conclusion, the study highlights the effectiveness of using PEFT-based quantized models in fine-tuning LLMs for automatic grading and feedback generation tasks. The results show promising accuracy rates and high similarity with expert feedback, indicating the potential for these models to be used in real-world educational settings. As technology continues to advance, it is exciting to see how these developments will shape the future of education and assessment methods.