{"id":{"repo_id":"bu","oai_identifier":"oai:open.bu.edu:2144/53394"},"canonical_url":"https://search.ndltd.org/etd/bu/oai:open.bu.edu:2144/53394","repository":{"repo_id":"bu","name":"Boston University","base_url":"https://open.bu.edu/oai/request"},"display":{"title":"Toward equitable and affect-enhanced face analysis and generation","abstract":"Research in face analysis and generation has made substantial progress, enabling applications in content generation, human behavior understanding, and human-computer interaction. However, significant challenges remain, including: (1) persistent racial and gender biases in both discriminative and generative face models due to imbalanced datasets; (2) limited fine-grained expressivity in face generation, particularly in producing stylized faces with nuanced facial expressions; and (3) insufficient integration of facial affective and multimodal behavioral signals for effective prediction of human behavior, especially in educational contexts. This dissertation addresses these gaps by introducing novel datasets and frameworks. First, we develop KenyanFace, a carefully curated face dataset designed to address racial and skin color biases inherent in existing face datasets and models. Next, we enhance diffusion-based text-to-image (T2I) generative frameworks to produce stylized cartoon faces with nuanced, fine-grained facial expressions. Furthermore, we present a series of datasets and models for effectively integrating facial cues with multimodal behavioral signals to predict complex human behaviors such as student engagement and learning outcomes. We collect and annotate datasets comprising facial videos and learning log data recorded during student interactions with an intelligent tutoring system. Leveraging these datasets, we propose several novel affective models that accurately predict student engagement and learning outcomes. Specifically, we introduce models utilizing (1) affective transfer learning methods to effectively incorporate facial affective features, and (2) multimodal approaches that integrate both facial features and learning log data. Building upon these predictive models, we further develop algorithms for personalized selection of educational content tailored to students' real-time emotional and cognitive states, thereby enhancing adaptive interventions and improving learning outcomes.","abstract_html":"Research in face analysis and generation has made substantial progress, enabling applications in content generation, human behavior understanding, and human-computer interaction. However, significant challenges remain, including: (1) persistent racial and gender biases in both discriminative and generative face models due to imbalanced datasets; (2) limited fine-grained expressivity in face generation, particularly in producing stylized faces with nuanced facial expressions; and (3) insufficient integration of facial affective and multimodal behavioral signals for effective prediction of human behavior, especially in educational contexts. This dissertation addresses these gaps by introducing novel datasets and frameworks. First, we develop KenyanFace, a carefully curated face dataset designed to address racial and skin color biases inherent in existing face datasets and models. Next, we enhance diffusion-based text-to-image (T2I) generative frameworks to produce stylized cartoon faces with nuanced, fine-grained facial expressions. Furthermore, we present a series of datasets and models for effectively integrating facial cues with multimodal behavioral signals to predict complex human behaviors such as student engagement and learning outcomes. We collect and annotate datasets comprising facial videos and learning log data recorded during student interactions with an intelligent tutoring system. Leveraging these datasets, we propose several novel affective models that accurately predict student engagement and learning outcomes. Specifically, we introduce models utilizing (1) affective transfer learning methods to effectively incorporate facial affective features, and (2) multimodal approaches that integrate both facial features and learning log data. Building upon these predictive models, we further develop algorithms for personalized selection of educational content tailored to students&#x27; real-time emotional and cognitive states, thereby enhancing adaptive interventions and improving learning outcomes.","abstract_has_math":false,"creators":["Yu, Hao"],"institution":"Boston University","degree_name":"Doctor of Philosophy","degree_level":"doctoral","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":[],"advisors":["Betke, Margrit"],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025","date_published":"2025","updated_at":"2026-11-18T20:27:37Z","subjects":["Computer science"],"languages":["en_US"],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[{"key":"dc:identifier.orcid","label":"ORCID","values":["0000-0002-4334-604X"],"render_values":[{"text":"0000-0002-4334-604X","href":"https://orcid.org/0000-0002-4334-604X","code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/2144/53394","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"https://hdl.handle.net/2144/53394","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://open.bu.edu/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Aopen.bu.edu%3A2144%2F53394","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Betke, Margrit"]},{"key":"dc:creator","label":"Author","values":["Yu, Hao"]},{"key":"dc:identifier.orcid","label":"ORCID","values":["0000-0002-4334-604X"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2026-07-29T19:03:56Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2026-07-29T19:03:56Z"]},{"key":"dc:date.issued","label":"Date","values":["2025"]},{"key":"dc:date.updated","label":"Dc Date Updated","values":["2026-07-21T19:16:40Z"]},{"key":"dc:type","label":"Dc Type","values":["Thesis/Dissertation"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Boston University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer science"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/2144/53394"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["2025"]},{"key":"dc:description.abstract","label":"Abstract","values":["Research in face analysis and generation has made substantial progress, enabling applications in content generation, human behavior understanding, and human-computer interaction. However, significant challenges remain, including: (1) persistent racial and gender biases in both discriminative and generative face models due to imbalanced datasets; (2) limited fine-grained expressivity in face generation, particularly in producing stylized faces with nuanced facial expressions; and (3) insufficient integration of facial affective and multimodal behavioral signals for effective prediction of human behavior, especially in educational contexts. This dissertation addresses these gaps by introducing novel datasets and frameworks. First, we develop KenyanFace, a carefully curated face dataset designed to address racial and skin color biases inherent in existing face datasets and models. Next, we enhance diffusion-based text-to-image (T2I) generative frameworks to produce stylized cartoon faces with nuanced, fine-grained facial expressions. Furthermore, we present a series of datasets and models for effectively integrating facial cues with multimodal behavioral signals to predict complex human behaviors such as student engagement and learning outcomes. We collect and annotate datasets comprising facial videos and learning log data recorded during student interactions with an intelligent tutoring system. Leveraging these datasets, we propose several novel affective models that accurately predict student engagement and learning outcomes. Specifically, we introduce models utilizing (1) affective transfer learning methods to effectively incorporate facial affective features, and (2) multimodal approaches that integrate both facial features and learning log data. Building upon these predictive models, we further develop algorithms for personalized selection of educational content tailored to students' real-time emotional and cognitive states, thereby enhancing adaptive interventions and improving learning outcomes."]},{"key":"dc:title","label":"Title","values":["Toward equitable and affect-enhanced face analysis and generation"]}]}],"canonical_facts":{"dc:contributor.advisor":["Betke, Margrit"],"dc:creator":["Yu, Hao"],"dc:date.accessioned":["2026-07-29T19:03:56Z"],"dc:date.available":["2026-07-29T19:03:56Z"],"dc:date.issued":["2025"],"dc:date.updated":["2026-07-21T19:16:40Z"],"dc:description":["2025"],"dc:description.abstract":["Research in face analysis and generation has made substantial progress, enabling applications in content generation, human behavior understanding, and human-computer interaction. However, significant challenges remain, including: (1) persistent racial and gender biases in both discriminative and generative face models due to imbalanced datasets; (2) limited fine-grained expressivity in face generation, particularly in producing stylized faces with nuanced facial expressions; and (3) insufficient integration of facial affective and multimodal behavioral signals for effective prediction of human behavior, especially in educational contexts. This dissertation addresses these gaps by introducing novel datasets and frameworks. First, we develop KenyanFace, a carefully curated face dataset designed to address racial and skin color biases inherent in existing face datasets and models. Next, we enhance diffusion-based text-to-image (T2I) generative frameworks to produce stylized cartoon faces with nuanced, fine-grained facial expressions. Furthermore, we present a series of datasets and models for effectively integrating facial cues with multimodal behavioral signals to predict complex human behaviors such as student engagement and learning outcomes. We collect and annotate datasets comprising facial videos and learning log data recorded during student interactions with an intelligent tutoring system. Leveraging these datasets, we propose several novel affective models that accurately predict student engagement and learning outcomes. Specifically, we introduce models utilizing (1) affective transfer learning methods to effectively incorporate facial affective features, and (2) multimodal approaches that integrate both facial features and learning log data. Building upon these predictive models, we further develop algorithms for personalized selection of educational content tailored to students' real-time emotional and cognitive states, thereby enhancing adaptive interventions and improving learning outcomes."],"dc:identifier.orcid":["0000-0002-4334-604X"],"dc:identifier.uri":["https://hdl.handle.net/2144/53394"],"dc:language.iso":["en_US"],"dc:subject":["Computer science"],"dc:title":["Toward equitable and affect-enhanced face analysis and generation"],"dc:type":["Thesis/Dissertation"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["doctoral"],"thesis:degree_name":["Doctor of Philosophy"],"thesis:institution_name":["Boston University"]},"updated_at":"2026-11-18T20:27:37Z"}