Back to search

University of British Columbia

Controllable student engagement recognition system : from modular vision pipelines embedding-aware, class-balance-tunable contrastive learning

Abstract

dc:description

Automated student engagement recognition supports adaptive online learning, but current sys tems face two challenges. First, visual cues of engagement (face presence, facial affect, and head pose) must be extracted reliably. Second, naturally collected data are severely imbal anced. Minority classes, such as Highly Disengaged and Highly Engaged, are rare, so high overall accuracy can hide poor recognition of these classes. This thesis investigates both problems. First, it introduces a three-stage modular pipeline (face detection, facial emotion recognition, and head-pose estimation) with four configurations tested on the DAiSEE benchmark for binary engagement classification. The best configuration achieves 90.25% accuracy and a binary F1-score of 0.9485. Because binary DAiSEE is 94.26% Engaged, these aggregate scores do not establish reliable Not Engaged detection. The study instead provides a controlled, interpretable comparison of cue-extraction modules. Second, the analysis shifts to the four-class ordinal CMOSE benchmark and representation level training. DUAL-POOL MOCORANK (DP-MR) replaces MocoRank’s single contrastive score pool with two pools: a class-balanced pool representing each engagement class equally and a natural-prior pool preserving the original class distribution. Embedding-aware post-hoc calibration (EMBED-LR) uses a held-out validation split to adjust the model without retraining. On CMOSE, calibration moves the model from 77.64% accuracy and 52.44%averageper-class accuracy to more balanced results: 71.50%/62.87% for general use, 65.93%/62.69% for flagging at-risk students, and 64.29%/64.37% at the most balanced setting. Bootstrap analysis shows that these changes are statistically reliable. In a matched single-model comparison, DP-MR improves on the published MocoRank with Center Loss by +1.88 percentage points in average per-class accuracy. Together, these contributions show that engagement recognition should not be reported as one fixed result. DP-MR can favor either overall accuracy or better detection of rare but impor tant disengagement classes. No setting is clearly better than the strongest published average per-class accuracy (60.94%). The main contribution is the ability to choose the best setting for the task, supported by an independent MocoRank reimplementation with documented replication details.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Computer Science
Grantor dc:publisher
University of British Columbia
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Salem, Mahmoud Alaa
Contributors dc:contributor
  • Mohamed, Abdallah
  • Irani, Pourang

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/95240

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Salem, Mahmoud Alaa. Controllable student engagement recognition system : from modular vision pipelines embedding-aware, class-balance-tunable contrastive learning. master's thesis, University of British Columbia, 2026. http://hdl.handle.net/2429/95240