Back to search

The Open University

Modelling Alignment and Key Information for Automatic Grading

Abstract

dc:description.abstract

Automatic Short Answer Grading (ASAG) is the task of grading short answer questions using computer models, especially machine learning and artificial intelligence. Short answer questions differ from essay questions, as the required short answer response is usually about a sentence in length, whereas for essay questions, students are expected to elaborate in detail on the subject. Also, unlike multiple-choice questions, for short answer questions, students are not provided with choices from which to select a correct response. For short answer questions, students write the response in <i>free-text</i> or <i>natural language</i>. Since students can formulate the response in variety of ways, a single short answer question can have multiple forms of correct responses. This makes the ASAG task a very challenging one. <br></br><br></br> This dissertation introduces the notion of <i>key information</i> for semantic text similarity in the context of ASAG. We define key information as new information that is not present in the question, and which is essential for the answer to be correct. We propose an algorithm for identifying key information and develop a neural structured alignment model which uses the key information to accurately grade student responses. We test the model on the Beetle and SciEntsBank corpora against a range of state-of-the-art models, with our model outperforming the state-of-the-art models. <br></br><br></br> The primary contribution of this dissertation is a set of methods that use semantic similarity techniques to obtain state-of-the-art results in the ASAG task. We first investigate how well existing neural network models work on the ASAG task. We look at word alignment and structured alignment attention networks. We then show how the performance of these architectures can be improved by augmenting them with key information. I present two different approaches to learning key information. The first approach uses an intuitive algorithm to learn the key information from a question’s reference answers. The key information is then passed to the neural network as input. The second approach uses a complex-number model to learn key information within the neural network itself.

Degree

thesis:*
Name dc:type.qualificationname
phd
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
The Open University
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Pandey, Suraj Jung

Rights

Language dc:language
en

Chain of custody

source
Harvested from
The Open University
Base URL
oro.open.ac.uk/cgi/oai2
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Pandey, Suraj Jung. Modelling Alignment and Key Information for Automatic Grading. doctoral thesis, The Open University, 2022.