Back to search

University of South Florida

Multimodal Emotion Recognition Using 3D Facial Landmarks, Action Units, and Physiological Data

Abstract

dc:description

To fully understand the complexities of human emotion, the integration of multiple physical features from different modalities can be advantageous. Considering this, this thesis presents an approach to emotion recognition using handcrafted features that consist of 3D facial data, action units, and physiological data. Each modality independently, as well as the combination of each for recognizing human emotion were analyzed. This analysis includes the use of principal component analysis to determine which dimensions of the feature vector are most important for emotion recognition. The proposed features are shown to be able to be used to accurately recognize emotion and that the proposed approach outperforms the current state of the art on the BP4D+ dataset, across multiple modalities.

Degree

thesis:*
Name thesis:degree_name
Master of Science (M.S.)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Computer Science and Engineering
Grantor
University of South Florida
Year dc:date
2019

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Fabiano, Diego
Contributors dc:contributor
  • Shaun Canavan, Ph.D.
  • Tempestt Neal, Ph.D.
  • Paul Rosen, Ph.D.

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.usf.edu/etd/8025

Chain of custody

source
Harvested from
University of South Florida
Base URL
digitalcommons.usf.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Fabiano, Diego. Multimodal Emotion Recognition Using 3D Facial Landmarks, Action Units, and Physiological Data. Thesis thesis, University of South Florida, 2019. https://digitalcommons.usf.edu/etd/8025