National and Kapodistrian University of Athens
Facial expression retrieval using 3-dimensional mesh sequences
Abstract
dc:descriptionIn recent years, the increased availability of inexpensive 3D object acquisition hardware and simplified 3D modeling software has resulted in the creation of massive 3D facial expression mesh sequences datasets that are either publicly available or for proprietary use. Consiquently, two new problems for the research community arose: Facial Expression Recognition from 3D mesh sequences and Facial Expression Retrieval from 3D mesh sequences. The first problem has gained a lot of interest among the research community. On the contrary, no sufficient research has been conducted on the second problem that deals with Retrieval.This dissertation focuses on the problem of facial expression retrieval from large datasets of 3D facial expressions mesh sequences. In order to address this problem we develop a 3-step retrieval scheme: (i) initially, eight 3D facial landmarks are automatically detected on each 3D face mesh of the sequence. (ii) Next, the landmarks extracted in the previous step, are used in order for the descriptors of the 3D facial expression mesh sequence to be created. (iii) Finally, appropriate distance functions are used in order for different descriptors (i.e. query descriptor vs dataset descriptor) to be compared and the retrieval list is produced. The core of the problem is the creation of appropriate descriptors. Six pioneer descriptors were created during this dissertation for 3D mesh sequence facial expression retrieval purposes (GeoTopo, GeoTopo+, DCT-GeoTopo, WT-GeoTopo+, CVD, WT-CVD). The aforementioned descriptors steadily increased the retrieval evaluation metrics. Two of them are spatial, which means that they are based only on spatial modifications of the facial expressions across time, and the remaining four are spatio-temporal, which means that they are based on both time and space modifications of the facial expressions.GeoTopo is a hybrid spatial descriptor which captures the topological as well as the geometric information of the 3D face meshes along time. This is achieved by concatenating two different sub-descriptors, one for the topology and one for the geometry of the 3D face mesh. GeoTopo+ is a hybrid spatial descriptor and is an improved version of GeoTopo. GeoTopo+ used two sub-descriptors for capturing the facial geometry and one for capturing facial topology.The motivation behind the proposed spatial, hybrid facial expression descriptors is the fact that some facial expressions, like happiness and surprise, are characterized by obvious changes in the mouth topology while others, like anger, fear and sadness, produce geometric but no significant topological changes.DCT-GeoTopo is the first attempt of constructing a spatio-temporal descriptor for 3D mesh sequence facial expression retrieval purposes. Initially, DCT-GeoTopo descriptor captures topological information of the 3D facial expression sequence. In the sequel, Discrete Cosine Transformation is applied on the aforementioned information consulting to the final spatio-temporal descriptor. WT-GeoTopo+ is a hybrid spatio-temporal descriptor which captures the geometric and the topological information of the 3D meshes in a similar way as GeoTopo+ does. In the sequel, the aforementioned spatial information is filtered by using Wavelet Transformation resulting to our final spatio-temporal descriptor. CVD descriptor is a spatio-temporal descriptor which exploits the depth information of the eight chosen facial landmarks. Finally, WT-CVD is an improved version of CVD which is produced after performing the Wavelet Transformation, on the depth information of the extracted facial landmarks.The motivation behind the proposed spatio-temporal descriptors is the fact that, in general, spatio-temporal descriptors are much more frugal, in terms of space and time requirements, than spatial descriptors. In addition, spatio-temporal descriptors are invariant to the number of the 3D face meshes of a facial expression sequence.The descriptors developed and described in this dissertation are evaluated in terms of retrieval accuracy and demonstrated using both quantitative and qualitative measures via an extensive consistent evaluation against state-of-the-art descriptors on standard datasets. This comparison illustrates the superiority of our descriptors compared to the state-of-the-art ones. Furthermore, a technique which exploits the retrieval results, in order to achieve unsupervised facial expression recognition from 3D mesh sequences, is presented. Our proposed technique achieve better results in terms of classification accuracy compared to the supervised dynamic 3D facial expression recognition state-of-the-art techniques.
Degree
thesis:*- Grantor dc:publisher
- National and Kapodistrian University of Athens
- Year dc:date
- 2016
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Danelakis, Antonios
- Δανελάκης, Αντώνιος
Subjects
dc:subject × 16- Δυναμική τριδιάστατη πλεγματοσειρά
- Aνάκτηση αντικειμένου
- Κυρτότητα
- Υπογραφή θερμότητας κελύφους
- Κανονικά διανύσματα
- Μετασχηματισμός συνημίτονου/κυματιδίων
- Dynamic 3D mesh sequence
- Object retrieval
- Curvature
- Heat kernel signature
- Normal vectors
- Cosine/Wavelet transformation
- Φυσικές Επιστήμες
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Natural Sciences
- Computer and Information Sciences
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/38226