Institutes outside Greece
Smart CAT: machine learning for configurable assessments in serious games
Abstract
dc:descriptionSerious games being at the forefront of digital education provide new opportunities for active learning and can effectively support the development of complex competencies that are considered to be critical for the people of the 21st century. Nonetheless, assessing such competencies with standard tests (e.g. multiple choice questionnaires) is a hard, if not impossible, task. Therefore, developing new assessment tools that hold educational value and rely on principled methodologies is of great importance. In this thesis, we utilize a data-driven assessment methodology called Stealth Assessment (SA) and develop an assessment software tool based on it for serious games. SA is an assessment methodology that uses (a) Evidence Centered Design to allow the configuration of assessments on different competence structures and (b) Machine Learning technology to produce inferences regarding the learners’ mastery levels on respective competencies. Despite SA being a proven methodology, its application is still complex, laborious and time-consuming. To address this issue, this thesis presents a framework for the development and implementation of a software tool, called Smart CAT (Smart Configurable Assessment Tool), which lowers the barriers of using SA and accommodates its wider application. Smart CAT allows for the flexible arrangement of data-driven assessment through its software wizard (which is enhanced by a respective GUI) and automatically executes ML-related functions in its back-end subsystem. Smart CAT was examined for its robustness using a simulation approach and it was found that it was able to effectively cope with a set of different test conditions. Also, when tested for empirical validity with real-world data from two separate serious games, Smart CAT was found to be able to provide accurate and valid assessments, given reliable input materials (data and statistical model). In addition, when examined for its overall usability, Smart CAT’s was found to be useful, easy to use, easy to learn, and satisfactory for its end-users. The outcomes of this thesis show that Smart CATeffectively mitigates the arrangement of SA, and that it can already be applied in education and research. However, future work is needed to upgrade its functionality and usability to turn it into a full-fledged market-ready product. To sum up, this thesis provides a novel perspective on data-driven assessments in serious games and contributes substantially to their practicability by combining educational methods and practices with cut-edging technology. Thereby, it adds to furthering the educational value of serious games and therefore promoting the use of serious games in mainstream education.
Degree
thesis:*- Grantor dc:publisher
- Institutes outside Greece
- Year dc:date
- 2020
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Georgiadis, Konstantinos
- Γεωργιάδης, Κωνσταντίνος
Subjects
dc:subject × 14- Αξιολόγηση μαθητών
- Τεχνιτή νοημοσύνη
- Μηχανική εκμάθηση
- Εκπαιδευτικά παιχνίδια
- Student assessment
- Artificial intelligence
- Machine learning
- Serious games
- Άλλες Επιστήμες Μηχανικού και Τεχνολογίες
- Επιστήμες Μηχανικού και Τεχνολογία
- Μηχανική, διεπιστημονική προσέγγιση
- Other Engineering and Technologies
- Engineering and Technology
- Engineering, interdisciplinary
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/55310