Democritus University of Thrace (DUTH)
Deep representation learning with application in multimodal emotion analysis for robotics
Abstract
dc:descriptionOne of the most prominent attributes of Neural Networks (NNs) constitutes their capability of learning to extract robust and descriptive features from high-dimensional data, like images. Hence, such an ability renders their exploitation as feature extractors particularly frequent in an abundance of modern reasoning systems. Their application scope mainly includes complex cascade tasks, like multi-modal recognition, deep Reinforcement Learning (RL), as well as their exploitation as descriptors in feature learning challenges, a field that enjoys apparent popularity over the past few years. Feature or representation learning focuses on the development of effective loss functions that ensure both high feature discrimination among different classes, as well as low geodesic distance between the feature vectors of a given class. The vast majority of the contemporary works rely their formulation on an empirical assumption (H) about the feature space (F) of a network’s last hidden layer, claiming that the weight vector of a class accounts for its geometrical center in the studied space. However, NNs induce implicit biases that are difficult to avoid or to deal with and are not met in traditional image descriptors. Moreover, the lack of knowledge for describing the intra-layer properties -and thus their general behavior- restricts the further applicability of the extracted features. A research field that can be considerably benefited by a robust feature extraction scheme is that of multi-modal emotion recognition. The advancement of Human-Robot Interaction (HRI) drives research into the development of advanced emotion identification architectures that fathom audio-visual (A-V) modalities of human emotion. State-of-the-art approaches exploit unimodal Deep Neural Networks (DNN) to process sensory inputs and learn a latent representation for each modality. Then, the extracted unimodal vectors are concatenated and fed into a fusion network that is responsible for the extraction of a compact emotional representation. However, the introduced methods in multi-modal emotion recognition mainly focus on the classification of complete video sequences, leading to systems with no online potentialities. Such techniques are capable of predicting emotions only when the videos are concluded, thus restricting their applicability in practical scenarios. The motivation of this study can be conceived through a bottom-up manner. In specific, beginning with the task to incorporate online capabilities in an A-V emotion recognition system, in order to render it suitable for HRI scenarios, the necessity of Summary and Contribution improving the adopted techniques in the fusion network emerged. Consequently, by investigating the possible strategies for fusing the A-V modalities, we came up with the challenge of understanding the properties of the representations learned by the unimodal feature extractors. The findings of the above analysis led to the conclusion that empirical assumptions such H can not be arbitrarily taken for granted. Hence, we were challenged to proceed with several adjustments regarding existing methods in the field of feature learning.
Degree
thesis:*- Grantor dc:publisher
- Democritus University of Thrace (DUTH)
- Year dc:date
- 2021
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kansizoglou, Ioannis
- Κανσίζογλου, Ιωάννης
Subjects
dc:subject × 20- Υπολογιστική όραση
- Αναγνώριση συναισθήματος
- Τεχνητή νοημοσύνη
- Επεξεργασία εικόνας βίντεο
- Ενισχυτική μάθηση
- Συνεργασία Ανθρώπου Ρομπότ
- Ρομποτική
- Computer vision
- Emotion recognition
- Artificial intelligence
- Image video processing
- Reinforcement learning
- Human-robot interaction
- Robotics
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/50435