{"id":{"repo_id":"usf","oai_identifier":"oai:digitalcommons.usf.edu:etd-9222"},"canonical_url":"https://search.ndltd.org/etd/usf/oai:digitalcommons.usf.edu:etd-9222","repository":{"repo_id":"usf","name":"University of South Florida","base_url":"https://digitalcommons.usf.edu/do/oai/"},"display":{"title":"Multimodal Emotion Recognition Using 3D Facial Landmarks, Action Units, and Physiological Data","abstract":"To fully understand the complexities of human emotion, the integration of multiple physical features from different modalities can be advantageous. Considering this, this thesis presents an approach to emotion recognition using handcrafted features that consist of 3D facial data, action units, and physiological data. Each modality independently, as well as the combination of each for recognizing human emotion were analyzed. This analysis includes the use of principal component analysis to determine which dimensions of the feature vector are most important for emotion recognition. The proposed features are shown to be able to be used to accurately recognize emotion and that the proposed approach outperforms the current state of the art on the BP4D+ dataset, across multiple modalities.","abstract_html":"To fully understand the complexities of human emotion, the integration of multiple physical features from different modalities can be advantageous. Considering this, this thesis presents an approach to emotion recognition using handcrafted features that consist of 3D facial data, action units, and physiological data. Each modality independently, as well as the combination of each for recognizing human emotion were analyzed. This analysis includes the use of principal component analysis to determine which dimensions of the feature vector are most important for emotion recognition. The proposed features are shown to be able to be used to accurately recognize emotion and that the proposed approach outperforms the current state of the art on the BP4D+ dataset, across multiple modalities.","abstract_has_math":false,"creators":["Fabiano, Diego"],"institution":"University of South Florida","degree_name":"Master of Science (M.S.)","degree_level":"Thesis","degree_discipline":"Computer Science and Engineering","degree_department":null,"school":null,"contributors":["Shaun Canavan, Ph.D.","Tempestt Neal, Ph.D.","Paul Rosen, Ph.D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019-10-23T07:00:00Z","date_published":"2019-10-23T07:00:00Z","updated_at":"2026-11-19T08:51:02Z","subjects":["Affective","Classification","Detection","Expression","Face","Statistical model","Computer Sciences"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usf.edu/etd/8025","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usf.edu/etd/8025","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usf.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usf.edu%3Aetd-9222","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Shaun Canavan, Ph.D.","Tempestt Neal, Ph.D.","Paul Rosen, Ph.D."]},{"key":"dc:creator","label":"Author","values":["Fabiano, Diego"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2019-10-23T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science and Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis","M.S."]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science (M.S.)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of South Florida"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Affective","Classification","Detection","Expression","Face","Statistical model","Computer Sciences"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usf.edu/etd/8025"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["To fully understand the complexities of human emotion, the integration of multiple physical features from different modalities can be advantageous. Considering this, this thesis presents an approach to emotion recognition using handcrafted features that consist of 3D facial data, action units, and physiological data. Each modality independently, as well as the combination of each for recognizing human emotion were analyzed. This analysis includes the use of principal component analysis to determine which dimensions of the feature vector are most important for emotion recognition. The proposed features are shown to be able to be used to accurately recognize emotion and that the proposed approach outperforms the current state of the art on the BP4D+ dataset, across multiple modalities."]},{"key":"dc:title","label":"Title","values":["Multimodal Emotion Recognition Using 3D Facial Landmarks, Action Units, and Physiological Data"]}]}],"canonical_facts":{"dc:contributor":["Shaun Canavan, Ph.D.","Tempestt Neal, Ph.D.","Paul Rosen, Ph.D."],"dc:creator":["Fabiano, Diego"],"dc:date":["2019-10-23T07:00:00Z"],"dc:description":["To fully understand the complexities of human emotion, the integration of multiple physical features from different modalities can be advantageous. Considering this, this thesis presents an approach to emotion recognition using handcrafted features that consist of 3D facial data, action units, and physiological data. Each modality independently, as well as the combination of each for recognizing human emotion were analyzed. This analysis includes the use of principal component analysis to determine which dimensions of the feature vector are most important for emotion recognition. The proposed features are shown to be able to be used to accurately recognize emotion and that the proposed approach outperforms the current state of the art on the BP4D+ dataset, across multiple modalities."],"dc:identifier":["https://digitalcommons.usf.edu/etd/8025"],"dc:subject":["Affective","Classification","Detection","Expression","Face","Statistical model","Computer Sciences"],"dc:title":["Multimodal Emotion Recognition Using 3D Facial Landmarks, Action Units, and Physiological Data"],"thesis:degree_discipline":["Computer Science and Engineering"],"thesis:degree_level":["Thesis","M.S."],"thesis:degree_name":["Master of Science (M.S.)"],"thesis:institution_name":["University of South Florida"]},"updated_at":"2026-11-19T08:51:02Z"}