Back to search

Democritus University of Thrace (DUTH)

Deep representation learning with application in multimodal emotion analysis for robotics

Abstract

dc:description

One of the most prominent attributes of Neural Networks (NNs) constitutes their capability of learning to extract robust and descriptive features from high-dimensional data, like images. Hence, such an ability renders their exploitation as feature extractors particularly frequent in an abundance of modern reasoning systems. Their application scope mainly includes complex cascade tasks, like multi-modal recognition, deep Reinforcement Learning (RL), as well as their exploitation as descriptors in feature learning challenges, a field that enjoys apparent popularity over the past few years. Feature or representation learning focuses on the development of effective loss functions that ensure both high feature discrimination among different classes, as well as low geodesic distance between the feature vectors of a given class. The vast majority of the contemporary works rely their formulation on an empirical assumption (H) about the feature space (F) of a network’s last hidden layer, claiming that the weight vector of a class accounts for its geometrical center in the studied space. However, NNs induce implicit biases that are difficult to avoid or to deal with and are not met in traditional image descriptors. Moreover, the lack of knowledge for describing the intra-layer properties -and thus their general behavior- restricts the further applicability of the extracted features. A research field that can be considerably benefited by a robust feature extraction scheme is that of multi-modal emotion recognition. The advancement of Human-Robot Interaction (HRI) drives research into the development of advanced emotion identification architectures that fathom audio-visual (A-V) modalities of human emotion. State-of-the-art approaches exploit unimodal Deep Neural Networks (DNN) to process sensory inputs and learn a latent representation for each modality. Then, the extracted unimodal vectors are concatenated and fed into a fusion network that is responsible for the extraction of a compact emotional representation. However, the introduced methods in multi-modal emotion recognition mainly focus on the classification of complete video sequences, leading to systems with no online potentialities. Such techniques are capable of predicting emotions only when the videos are concluded, thus restricting their applicability in practical scenarios. The motivation of this study can be conceived through a bottom-up manner. In specific, beginning with the task to incorporate online capabilities in an A-V emotion recognition system, in order to render it suitable for HRI scenarios, the necessity of Summary and Contribution improving the adopted techniques in the fusion network emerged. Consequently, by investigating the possible strategies for fusing the A-V modalities, we came up with the challenge of understanding the properties of the representations learned by the unimodal feature extractors. The findings of the above analysis led to the conclusion that empirical assumptions such H can not be arbitrarily taken for granted. Hence, we were challenged to proceed with several adjustments regarding existing methods in the field of feature learning.

Degree

thesis:*
Grantor dc:publisher
Democritus University of Thrace (DUTH)
Year dc:date
2021

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Kansizoglou, Ioannis
  • Κανσίζογλου, Ιωάννης

Subjects

dc:subject × 20

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/50435

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Kansizoglou, Ioannis; Κανσίζογλου, Ιωάννης. Deep representation learning with application in multimodal emotion analysis for robotics. Democritus University of Thrace (DUTH), 2021. http://hdl.handle.net/10442/hedi/50435