University of North Florida. School of Computing
Automated short-answer grading and misconception detection using large language models
Abstract
dc:description.abstractAs education technology continues to evolve, the domains of Automatic Short-Answer Grading (ASAG) and Automated Misconception Detection (AMD) stand at the forefront of innovative approaches to educational assessment. We explore the transformative potential of Large Language Models (LLMs) in revolutionizing these critical areas. Leveraging the remarkable capabilities of LLMs in semantic inference, contextual understanding, and transfer learning, we embark on a comprehensive journey to enhance both ASAG and AMD. On ASAG, we illuminate the efficacy of transfer learning by fine-tuning RoBERTa Large, a state-of-the-art LLM, on task-related corpora, e.g. the Multi-Genre Natural Language Inference (MNLI) corpus. The model's adaptability across unseen questions and domains on the minority class, coupled with its narrowed performance gap from unseen answers, highlights the profound impact of transfer learning on grading diverse student responses. In the emerging realm of AMD, we pioneer a dataset and methodology that inaugurates a new era in misconception detection. Framing the task as Recognizing Textual Entailment (RTE), our approach with RoBERTa Large MNLI captures nuanced misconceptions, unveiling the untapped potential of LLMs in unraveling the intricate landscape of automated misconception detection. The synergy between these endeavors presents a holistic view of the transformative role anticipated for LLMs in automated educational assessment. Our research, spanning adaptability in short-answer grading and groundbreaking advancements in misconception detection, establishes a foundation for a future where LLMs excel not only in understanding nuanced student responses but also in pinpointing and rectifying misconceptions with unparalleled precision. These insights contribute significantly to the dynamic field of educational technology, heralding a new era wherein the full potential of LLMs is utilized to shape the trajectory of educational assessment.
Degree
thesis:*- Name thesis:degree_name
- Master of Science in Computer and Information Sciences (MS)
- Discipline thesis:degree_discipline
- Computing
- Grantor
- University of North Florida. School of Computing
- Date dc:date.issued
- Fall
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kazi, Nazmul H
- Kazi, Nazmul Hasan, 1993-
- Advisors dc:contributor.advisor
-
- Kahanda, Indika
- Liu, Xudong
- Prodanoff, Zornitza
- Contributors dc:contributor
-
- Dr. Indika Kahanda
- Dr. Xudong Liu
- Dr. Zornitza Prodanoff
Subjects
dc:subject × 22- Thesis
- University of North Florida
- UNF
- Dissertations, Academic -- UNF -- Master of Science in Computer and Information Sciences
- Dissertations, Academic -- UNF -- Computing
- Automatic Short-Answer Grading
- ASAG
- Automated Misconception Detection
- educational assessment
- Large Language Models
- LLM
- transfer learning
- NLI
- MNLI
- Recognizing Textual Entailment
- RTE
- BERT
- RoBERTa
- Artificial Intelligence and Robotics
- Grading and marking (Students) -- Automation -- Technological innovations; Questions and answers -- Data processing -- Technological innovations; Errors -- Data processing -- Technological innovations; Examinations -- Interpretation -- Automation -- Technological innovations; Educational evaluation -- Automation -- Technological innovations; Deep learning (Machine learning) -- Industrial applications; Education -- Examinations
- questions
- etc. -- Data processing -- Technological innovations; Transfer learning (Machine learning) -- Industrial applications; Artificial intelligence -- Data processing -- Industrial applications
Rights
dc:rightsIdentifiers
dc:identifier.*- Repository record dc:identifier
- https://digitalcommons.unf.edu/etd/1234