National Technical University of Athens (NTUA)
Leveraging sensor data for real time recognition of engagement in adaptive serious games for health
Abstract
dc:descriptionThe aim of the present Doctoral Thesis is the development of a novel conceptual framework for personalization in serious games (SGs) for health. The proposed framework leverages sensor data for the recognition of player engagement during interaction with SGs for health in real time. This approach aims to automatically generate game content based on player engagement, in-game performance, and health-related needs. In the present thesis, two novel SGs for health are designed and developed, aiming to promote self-health management in chronic health conditions and incorporating mechanics to facilitate procedural generation of content. A novel technique, based on a genetic algorithm, that employs heterogeneous data for procedural content generation (PCG) in SGs for health is presented. Two carefully designed experimental processes are implemented to investigate the feasibility of the proposed framework. The experimental processes collect data from sensors and interaction with the SG for health and investigate matters of player experience, educational value of the intervention, and efficiency of the proposed PCG technique. The two employed SGs for health aim to promote food and nutrition literacy and raise awareness and promote self-health management for obstructive sleep apnea, respectively. Analysis for sensor-based real-time recognition of engagement is conducted by approximating the ground truth, in terms of perceived engagement, through continuous annotations by participants. Results indicate that the educational value of the first SG for health is similar to a traditional intervention and demonstrate the predictive capacity of features extracted from the collected data towards perceived player engagement. In addition, statistically significant differences are revealed in terms of player experience in correlation with the generated content by the PCG. Furthermore, the PCG technique’s capacity to rely on clinically relevant sensor data to produce tailored game content is investigated in a pre-pilot study employing a SG for health for children with type 1 diabetes and/or obesity. Results display the PCG’s effectiveness in generating useful and relevant content, tailored to player needs, based on sensor and platform interaction data.Finally, the proposed PCG technique is evaluated with the use of deep reinforcement learning (DRL) agents for automated playing. Results indicate an overall superiority in DRL agents’ training when exposed to SG content produced by the proposed PCG technique. Overall findings included in the present Doctoral Thesis advocate towards the feasibility of the introduced conceptual framework. The insights gained from the experimental processes provide convincing arguments towards creating a closed real-time engagement feedback loop in adaptive SG for health, based on sensor and interaction data.
Degree
thesis:*- Grantor dc:publisher
- National Technical University of Athens (NTUA)
- Year dc:date
- 2023
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Mitsis, Konstantinos
- Μήτσης, Κωνσταντίνος
Subjects
dc:subject × 20- Παιχνίδια σοβαρού σκοπού
- Υγεία
- Προσαρμοστικότητα
- Αυτοματοποιημένη παραγωγή περιεχομένου
- Αισθητήρες
- Αναγνώριση
- Προσήλωση
- Serious games
- Health
- Adaptivity
- Procedural content generation
- Sensors
- Real time recognition
- Engagement
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/53713