{"id":{"repo_id":"usf","oai_identifier":"oai:digitalcommons.usf.edu:etd-9990"},"canonical_url":"https://search.ndltd.org/etd/usf/oai:digitalcommons.usf.edu:etd-9990","repository":{"repo_id":"usf","name":"University of South Florida","base_url":"https://digitalcommons.usf.edu/do/oai/"},"display":{"title":"Analysis of Contextual Emotions Using Multimodal Data","abstract":"Affective computing builds and evaluates systems that can recognize, interpret, and simulate human emotion. It is an interdisciplinary field, which includes computer science, psychology, and many others. For years, human emotion has been studied in psychology but recently has become a prominent field in computer science. Largely, the field of affective computing has been focused on analyzing static facial expressions to recognize human emotions, without taking bias (e.g. gender, data bias), context, or temporal information into account. Psychology has shown the difficulty of analyzing emotions without incorporating this type of information. In this dissertation, we have proposed new approaches to recognizing emotions by incorporating both contextual and temporal information, as well as approaches to mitigate data bias. More specifically, this dissertation has the following theoretical and application-based contributions: (1) The first work to recognize context using temporal dynamics from facial action units; (2) We recognize multiple self-reported emotions using facial expression-based videos; (3) A new approach to mitigate data bias in facial action units is proposed; and (4) Multimodal, temporal fusion of physiological signals and action units are used for emotion recognition. This dissertation has a wide range of applications in the fields including, but not limited to, medicine, security, and entertainment.","abstract_html":"Affective computing builds and evaluates systems that can recognize, interpret, and simulate human emotion. It is an interdisciplinary field, which includes computer science, psychology, and many others. For years, human emotion has been studied in psychology but recently has become a prominent field in computer science. Largely, the field of affective computing has been focused on analyzing static facial expressions to recognize human emotions, without taking bias (e.g. gender, data bias), context, or temporal information into account. Psychology has shown the difficulty of analyzing emotions without incorporating this type of information. In this dissertation, we have proposed new approaches to recognizing emotions by incorporating both contextual and temporal information, as well as approaches to mitigate data bias. More specifically, this dissertation has the following theoretical and application-based contributions: (1) The first work to recognize context using temporal dynamics from facial action units; (2) We recognize multiple self-reported emotions using facial expression-based videos; (3) A new approach to mitigate data bias in facial action units is proposed; and (4) Multimodal, temporal fusion of physiological signals and action units are used for emotion recognition. This dissertation has a wide range of applications in the fields including, but not limited to, medicine, security, and entertainment.","abstract_has_math":false,"creators":["Hinduja, Saurabh"],"institution":"University of South Florida","degree_name":"Doctor of Philosophy (Ph.D.)","degree_level":"Dissertation","degree_discipline":"Computer Science and Engineering","degree_department":null,"school":null,"contributors":["Shaun Canavan, Ph.D.","Rangachar Kasturi, Ph.D.","Jeffrey F. Cohn, Ph.D.","Marvin J. Andujar, Ph.D.","Pei-Sung Lin, Ph.D.","Elizabeth Schotter, Ph.D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021-03-31T07:00:00Z","date_published":"2021-03-31T07:00:00Z","updated_at":"2026-11-19T08:45:24Z","subjects":["Action Units","Expression","Physiological Signals","Computer Sciences"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usf.edu/etd/8793","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usf.edu/etd/8793","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usf.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usf.edu%3Aetd-9990","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Shaun Canavan, Ph.D.","Rangachar Kasturi, Ph.D.","Jeffrey F. Cohn, Ph.D.","Marvin J. Andujar, Ph.D.","Pei-Sung Lin, Ph.D.","Elizabeth Schotter, Ph.D."]},{"key":"dc:creator","label":"Author","values":["Hinduja, Saurabh"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2021-03-31T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science and Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation","Ph.D."]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy (Ph.D.)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of South Florida"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Action Units","Expression","Physiological Signals","Computer Sciences"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usf.edu/etd/8793"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Affective computing builds and evaluates systems that can recognize, interpret, and simulate human emotion. It is an interdisciplinary field, which includes computer science, psychology, and many others. For years, human emotion has been studied in psychology but recently has become a prominent field in computer science. Largely, the field of affective computing has been focused on analyzing static facial expressions to recognize human emotions, without taking bias (e.g. gender, data bias), context, or temporal information into account. Psychology has shown the difficulty of analyzing emotions without incorporating this type of information. In this dissertation, we have proposed new approaches to recognizing emotions by incorporating both contextual and temporal information, as well as approaches to mitigate data bias. More specifically, this dissertation has the following theoretical and application-based contributions: (1) The first work to recognize context using temporal dynamics from facial action units; (2) We recognize multiple self-reported emotions using facial expression-based videos; (3) A new approach to mitigate data bias in facial action units is proposed; and (4) Multimodal, temporal fusion of physiological signals and action units are used for emotion recognition. This dissertation has a wide range of applications in the fields including, but not limited to, medicine, security, and entertainment."]},{"key":"dc:title","label":"Title","values":["Analysis of Contextual Emotions Using Multimodal Data"]}]}],"canonical_facts":{"dc:contributor":["Shaun Canavan, Ph.D.","Rangachar Kasturi, Ph.D.","Jeffrey F. Cohn, Ph.D.","Marvin J. Andujar, Ph.D.","Pei-Sung Lin, Ph.D.","Elizabeth Schotter, Ph.D."],"dc:creator":["Hinduja, Saurabh"],"dc:date":["2021-03-31T07:00:00Z"],"dc:description":["Affective computing builds and evaluates systems that can recognize, interpret, and simulate human emotion. It is an interdisciplinary field, which includes computer science, psychology, and many others. For years, human emotion has been studied in psychology but recently has become a prominent field in computer science. Largely, the field of affective computing has been focused on analyzing static facial expressions to recognize human emotions, without taking bias (e.g. gender, data bias), context, or temporal information into account. Psychology has shown the difficulty of analyzing emotions without incorporating this type of information. In this dissertation, we have proposed new approaches to recognizing emotions by incorporating both contextual and temporal information, as well as approaches to mitigate data bias. More specifically, this dissertation has the following theoretical and application-based contributions: (1) The first work to recognize context using temporal dynamics from facial action units; (2) We recognize multiple self-reported emotions using facial expression-based videos; (3) A new approach to mitigate data bias in facial action units is proposed; and (4) Multimodal, temporal fusion of physiological signals and action units are used for emotion recognition. This dissertation has a wide range of applications in the fields including, but not limited to, medicine, security, and entertainment."],"dc:identifier":["https://digitalcommons.usf.edu/etd/8793"],"dc:subject":["Action Units","Expression","Physiological Signals","Computer Sciences"],"dc:title":["Analysis of Contextual Emotions Using Multimodal Data"],"thesis:degree_discipline":["Computer Science and Engineering"],"thesis:degree_level":["Dissertation","Ph.D."],"thesis:degree_name":["Doctor of Philosophy (Ph.D.)"],"thesis:institution_name":["University of South Florida"]},"updated_at":"2026-11-19T08:45:24Z"}