This thesis is dedicated to extracting expression robust features for 3D face recognition. The use of 3D imaging enables the extraction of discriminative features that can significantly improve the recognition performance due to the availability of facial surface information such as depth, surface normals and curvature. Expression robust analysis using information from both depth and surface normals is investigated by dividing the main facial region into patches of different scales. The nasal region and adjoining parts of the cheeks are utilized as they are more consistent over different expressions and are hard to deliberately occlude. In addition, in comparison with other parts of the face, these regions have a high potential to produce discriminative features for recognition and overcome pose variations. An overview and classification methodology of the widely used 3D face databases are first introduced to provide an appropriate reference for 3D face database selection. Using the FRGC and Bosphorus databases, a low complexity pattern rejector for expression robust 3D face recognition is proposed by matching curves on the nasal and its environs, which results in a low-dimension feature set of only 60 points. To extract discriminative features more locally, a novel multi-scale and multi-component local shape descriptor is further proposed, which achieves more competitive performances under the identification and verification scenarios. In contrast with many of the existing work on 3D face recognition that consider captures obtained with laser scanners or structured light, this thesis also investigates applications to reconstructed 3D captures from lower cost photometric stereo imaging systems that have applications in real-world situations. To this end, the performance of the expression robust face recognition algorithms developed for captures from laser scanners are further evaluated on the Photoface database, which contains naturalistic expression variations. To improve the recognition performance of all types of 3D captures, a universal landmarking algorithm is proposed that makes uses of different components of the surface normals. Using facial profile signatures and thresholded surface normal maps, facial roll and yaw rotations are calibrated and five main landmarks are robustly detected on the well-aligned 3D nasal region. The landmarking results show that the detected landmarks demonstrate high within-class consistency and can achieve good recognition performances under different expressions. This is also the first landmarking work specifically developed for the reconstructed 3D captures from photometric stereo imaging systems.

Ce travail de thèse concerne l’analyse de surfaces faciales en 3D, ainsi que leur traitement, dans le récent cadre de la modalité de reconnaissance de visages en 3D,basé sur des techniques d’appariement. Le traitement de la surface faciale et son analyse constituent une étape importante dans les algorithmes de reconnaissance de visage en 3D. La localisation de points d’intérêt anthropométriques du visage joue par ailleurs un rôle important dans les techniques de localisation du visage, de reconnaissance d’expression, de recalage, etc. Ainsi, leur localisation automatique joue un rôle crucial dans les algorithmes de traitement du visage 3D. Dans ce travail, nous avons mis l’accent sur la localisation précise et invariante en rotation des points d’intérêt, qui seront utilisés plus tard pour la reconnaissance de visages. Ces points d’intérêt sont localisés en combinant les propriétés locales de la surface faciale, exprimées en termes de géométrie différentielle, et un modèle global et générique du visage. Etant donné que la sensibilité des courbures, qui sont des propriétés de géométrie différentielle, au bruit, une des contributions de cette thèse est la modification d’une méthode de calcul de courbures. Cette modification incorpore le bruit de la surface dans la méthode de calcul, et permet de contrôler la progressivité des courbures. Par conséquent, nous pouvons localiser les points d’intérêt de la surface faciale avec précision et fiabilité (100% de bonnes localisation du bout du nez avec une erreur maximale de 8mmpar exemple) y compris en présence de rotations et de bruit. La modification de la méthode de calcul de courbure a été également testée pour différentes résolutions de visage, présentant des valeurs de courbure stables. Enfin, étant donné que donné que l’analyse de courbures mène à de nombreux candidats de points d’intérêt du visage, dont la validation est coûteuse, nous proposons de localiser les points d’intérêt grâce à une méthode d’apprentissage. Cette méthode permet de rejeter précocement des faux candidats avec une grande confiance, accélérant d’autant la localisation des points d’intérêt. La reconnaissance de visages à l’aide de modèles 3D est un sujet relativement nouveau, qui a été propose pour palier aux insuffisantes de la modalité de reconnaissance de visages en 2D. Cependant, les algorithmes de reconnaissance de visage en 3D sont généralement plus complexes. De plus, étant donné que les modèles de visage 3D décrivent la géométrie du visage, ils sont plus sensibles que les images 2Dde texture aux expressions faciales. Notre contribution est de réduire la dimensionnalité des données de départ en appariant les modèles de visage 3D au domaine 2Dà l’aide de méthodes, non rigides, d’appariement conformal. L’existence de modèles2D représentant les visages permet alors d’utiliser les techniques précédemment développées dans le domaine de la reconnaissance de visages en 2D. Dans nos travaux, nous avons utilisé les cartes conformales de visages 3D en conjonction avec l’algorithme2D2 PCA, atteignant le score de 86% en reconnaissance de rang 1 sur la base de données FRGC. L’efficacité de toutes les méthodes a été évaluée sur les bases FRGC et Bosphorus. / This Ph.D thesis work is dedicated to 3D facial surface analysis, processing as well as to the newly proposed 3D face recognition modality, which is based on mapping techniques. Facial surface processing and analysis is one of the most important steps for 3Dface recognition algorithms. Automatic anthropometric facial features localization also plays an important role for face localization, face expression recognition, face registration ect., thus its automatic localization is a crucial step for 3D face processing algorithms. In this work we focused on precise and rotation invariant landmarks localization, which are later used directly for face recognition. The landmarks are localized combining local surface properties expressed in terms of differential geometry tools and global facial generic model, used for face validation. Since curvatures, which are differential geometry properties, are sensitive to surface noise, one of the main contributions of this thesis is a modification of curvatures calculation method. The modification incorporates the surface noise into the calculation method and helps to control smoothness of the curvatures. Therefore the main facial points can be reliably and precisely localized (100% nose tip localization using 8 mm precision)under the influence of rotations and surface noise. The modification of the curvatures calculation method was also tested under different face model resolutions, resulting in stable curvature values. Finally, since curvatures analysis leads to many facial landmark candidates, the validation of which is time consuming, facial landmarks localization based on learning technique was proposed. The learning technique helps to reject incorrect landmark candidates with a high probability, thus accelerating landmarks localization. Face recognition using 3D models is a relatively new subject, which has been proposed to overcome shortcomings of 2D face recognition modality. However, 3Dface recognition algorithms are likely more complicated. Additionally, since 3D face models describe facial surface geometry, they are more sensitive to facial expression changes. Our contribution is reducing dimensionality of the input data by mapping3D facial models on to 2D domain using non-rigid, conformal mapping techniques. Having 2D images which represent facial models, all previously developed 2D face recognition algorithms can be used. In our work, conformal shape images of 3Dfacial surfaces were fed in to 2D2 PCA, achieving more than 86% recognition rate rank-one using the FRGC data set. The effectiveness of all the methods has been evaluated using the FRGC and Bosphorus datasets.

This thesis is dedicated to exploring the potential of the 3D shape of the nasal region for face recognition. In comparison to other parts of the face, the nose has a number of distinctive features that make it attractive for recognition purposes. It is relatively stable over different facial expressions, easy to detect because of its salient convexity, and difficult to be intentionally cover up without attracting suspicion. In addition compared to other facial parts, such as forehead, chin, mouth and eyes, the nose is not vulnerable to unintentional occlusions caused by scarves or hair. Prior to undertaking a thorough analysis of the discriminative features of the 3D nasal regions, an overview of denoising algorithms and their impact on the 3D face recognition algorithms is first provided. This analysis, which is one of the first to address this issue, evaluates the performance of 3D holistic algorithms when various denoising methods are applied. One important outcome of this evaluation is to determine the optimal denoising parameters in terms of the overall 3D face recognition performance. A novel algorithm is also proposed to learn the statistics of the noise generated by the 3D laser scanners and then simulate it over the face point clouds. Using this process, the denoising and 3D face recognition algorithms’ robustness over various noise powers can be quantitatively evaluated. A new algorithm is proposed to find the nose tip from various expressions and self-occluded samples. Furthermore, novel applications of the nose region to align the faces in 3D is provided through two pose correction methods. The algorithms are very consistent and robust against different expressions, partial and self-occlusions. The nose’s discriminative strength for 3D face recognition is analysed using two approaches. The first one creates its feature sets by applying nasal curves to the depth map. The second approach utilises a novel feature space, based on histograms of normal vectors to the response of the Gabor wavelets applied to the nasal region. To create the feature spaces, various triangular and spherical patches and nasal curves are employed, giving a very high class separability. A genetic algorithm (GA) based feature selector is then used to make the feature space more robust against facial expressions. The basis of both algorithms is a highly consistent and accurate nasal region landmarking, which is quantitatively evaluated and compared with previous work. The recognition ranks provide the highest identification performance ever reported for the 3D nasal region. The results are not only higher than the previous 3D nose recognition algorithms, but also better than or very close to recent results for whole 3D face recognition. The algorithms have been evaluated on three widely used 3D face datasets, FRGC, Bosphorus and UMB-DB.

Facial symmetry has been a central component in many studies on face perception. The relationship between bilateral symmetry and subjective judgments on faces is still arguable in the literature. In this study, a database of natural looking face images with different levels of symmetry is constructed using several digital preprocessing and morphing methods. Our aim is to investigate the correlations between quantified asymmetry, perceived symmetry and a subjective judgment: &lsquo / attractiveness&rsquo / . Images in the METU-Face Database are built to represent three levels of symmetry (original, intermediate, and symmetrical) within five classes which also represent the orientation of bilateral symmetry: left versus right. In addition, the asymmetry of original images is quantified using a landmark-based method. Based on the theory of holistic face perception, we introduce a novel method to quantify facial asymmetry wholesomely: Entropybased quantification. In Experiment 1 and 2, images were rated on attractiveness judgments and on perceived symmetry, respectively. Results indicate that landmark-based quantifications were not sufficient to account for perceived symmetry ratings (SRs), but they revealed that as the vertical deviation of the symmetry decreases, attractiveness rating (AR) collected from that face increases. Moreover, morphing classes and their relationship to both ARs and SRs were highly correlated. Consistent with the previously done research, symmetrical images were found more attractive. We found that although ARs were the same for left versus right composites, for SRs, there is a significant difference between left and right. Finally, a more elucidative quantification approach for subjective face perception is achieved through significant correlations of entropy scores with both ARs and SRs.

This Ph.D thesis work is dedicated to 3D facial surface analysis, processing as well as to the newly proposed 3D face recognition modality, which is based on mapping techniques. Facial surface processing and analysis is one of the most important steps for 3Dface recognition algorithms. Automatic anthropometric facial features localization also plays an important role for face localization, face expression recognition, face registration ect., thus its automatic localization is a crucial step for 3D face processing algorithms. In this work we focused on precise and rotation invariant landmarks localization, which are later used directly for face recognition. The landmarks are localized combining local surface properties expressed in terms of differential geometry tools and global facial generic model, used for face validation. Since curvatures, which are differential geometry properties, are sensitive to surface noise, one of the main contributions of this thesis is a modification of curvatures calculation method. The modification incorporates the surface noise into the calculation method and helps to control smoothness of the curvatures. Therefore the main facial points can be reliably and precisely localized (100% nose tip localization using 8 mm precision)under the influence of rotations and surface noise. The modification of the curvatures calculation method was also tested under different face model resolutions, resulting in stable curvature values. Finally, since curvatures analysis leads to many facial landmark candidates, the validation of which is time consuming, facial landmarks localization based on learning technique was proposed. The learning technique helps to reject incorrect landmark candidates with a high probability, thus accelerating landmarks localization. Face recognition using 3D models is a relatively new subject, which has been proposed to overcome shortcomings of 2D face recognition modality. However, 3Dface recognition algorithms are likely more complicated. Additionally, since 3D face models describe facial surface geometry, they are more sensitive to facial expression changes. Our contribution is reducing dimensionality of the input data by mapping3D facial models on to 2D domain using non-rigid, conformal mapping techniques. Having 2D images which represent facial models, all previously developed 2D face recognition algorithms can be used. In our work, conformal shape images of 3Dfacial surfaces were fed in to 2D2 PCA, achieving more than 86% recognition rate rank-one using the FRGC data set. The effectiveness of all the methods has been evaluated using the FRGC and Bosphorus datasets.

Localizada a aproximadamente 60 km da capital, Palmas-TO, está Porto Nacional, uma das cidades do estado do Tocantins que possui um significativo valor histórico, artístico, cultural e religioso para o antigo “norte goiano”, assim chamada a região norte para os que residiam na região sul, hoje estado de Goiás. Como podemos observar nos estudos de alguns autores que trataram sobre várias temáticas em Porto Nacional, como educação, patrimônio, história, a gênese da cidade de Porto Nacional se deu por três acontecimentos: a chegada dos Dominicanos, o surgimento de três jornais e a chegada de um médico à cidade. Vários são os motivos para ser considerada uma das cidades mais representativas do Estado, um deles é o fato de contar com uma forte tendência à religião, mas especificadamente o Catolicismo. Outro dado importante foi o tombamento, que aconteceu no ano de 2008, sendo a segunda cidade a ser tombada no Estado do Tocantins, e a primeira em território tocantinense. Traz uma retrospectiva sobre o processo de evolução do patrimônio com destaque as Cartas Patrimoniais de Atenas, em (1931), Carta de Veneza, de (1964), a Convenção Relativa à Proteção do Patrimônio Mundial Cultural e Natural, de (1972), e a declaração de Amsterdã, em (1975). Essas cartas foram de fundamental importância para a consolidação do Patrimônio Histórico e Cultural nos dias atuais. O estudo propõe discutir e refletir sobre ações que norteiam o Turismo Cultural na cidade. A primeira etapa que compõe o estudo foi a aplicação de um questionário aos turistas que visitaram a Catedral Nossa Senhora das Mercês e o Museu Histórico e Cultural no período de férias referente ao mês de julho e dezembro de 2013, e o processo de patrimonialização do Centro Histórico, segundo os argumentos e entrevistas feitas com os moradores. Foram aplicados aos turistas alguns questionários semiestruturados como primeira etapa da pesquisa. Um dos principais objetivos é discutir o desenvolvimento do turismo cultural na cidade para o desenvolvimento da atividade turística na cidade. A segunda etapa consiste em verificar a percepção dos moradores na poligonal de tombamento do centro histórico quanto ao processo de patrimonialização com base em documentos formais que constituem a Ata de Tombamento do Centro Histórico e o Dossiê de Tombamento. Faz-se uma reflexão sobre o atual contexto que a cidade se encontra. Assim, teremos duas visões, do turista e do morador, a fim de responder se o tombamento da cidade de Porto Nacional corresponde às expectativas do turista e do morador. Considerado o berço cultural do Patrimônio Histórico do Estado, com base em algumas questões contidas no questionário, surge a reflexão para o questionamento: a existência o Turismo cultural na cidade de Porto Nacional-TO bem como a participação dos moradores para o tombamento do centro histórico. / Placed close to 60 km from the state district, Palmas-TO, there is Porto Nacional, one of the Tocantins`state cities well known for its historical, artistic, cultural and religious value to the old times known as “norte goiano”, which is the north region for those who lived in the south, this last one is now called Goiás state. As we can observe through the study of some authors dedicated to several subjects on Porto Nacional, like Education, Historical heritage and History, Porto Nacional city genesis was set by three start points: the Dominicans arrival, the dawn of three newspapers and lastly the arrival of a physician to the city. There are many reasons why it is one of the most representative cities in the state, one of them is the strong tendency to religion, most of all the catholic one. Another important aspect was its landmarking by the historical heritage, which occurred in 2008. It is also needed to mention that it was the second city to be listed as historical heritage on the state and also the first in its territory. It carries an eventful hindsight of the historical heritage evolution process whereas it includes the Historical Heritage Chart of Athenas (1931), Venice Chart (1964), Convention for World Cultural and Natural Historical Heritage (1972) and the Amsterdan Declaration (1975). These charts were weighty to the historical heritage landmarks consolidation in our times. This study suggests to discuss and reflect on the actions that give north to the city cultural tourism. The first step of this study consisted in the application of a poll to the tourists that visited the city`s church and museum during the last 2013 July and December, as well as this part points out the historical heritage process of Porto Nacional`s historical center according to the arguments and interviews done upon the city dwellers. Some semi-structured polls were applied on the tourists as first step of this research. One of the standard points of this work is to measure cultural tourism importance to the city tourism as a whole. The second step of this work is to verify the perception of the nearby inhabitants of the city’s historical center on the historical heritage process based on formal documents as the Historical Center Heritage Landmarking Minute and the Historical Heritage Landmarking Dossier. This study drifts a reflection upon the actual context of the city. After all, this study contains two vision, the dweller and the tourist one, in order to answer if the city historical heritage listing replies to their expectations. Having in mind the fact that the city is considered the cultural birthplace of the state, and also based on some issues enlightened by the poll, a reflection arises: the existence of cultural tourism in Porto Nacional and the participation of its dwellers to the historical center landmarking.

This Ph.D thesis work is dedicated to automatic facial analysis in 3D, including facial landmarking and facial expression recognition. Indeed, facial expression plays an important role both in verbal and non verbal communication, and in expressing emotions. Thus, automatic facial expression recognition has various purposes and applications and particularly is at the heart of "intelligent" human-centered human/computer(robot) interfaces. Meanwhile, automatic landmarking provides aprior knowledge on location of face landmarks, which is required by many face analysis methods such as face segmentation and feature extraction used for instance for expression recognition. The purpose of this thesis is thus to elaborate 3D landmarking and facial expression recognition approaches for finally proposing an automatic facial activity (facial expression and action unit) recognition solution.In this work, we have proposed a Bayesian Belief Network (BBN) for recognizing facial activities, such as facial expressions and facial action units. A StatisticalFacial feAture Model (SFAM) has also been designed to first automatically locateface landmarks so that a fully automatic facial expression recognition system can be formed by combining the SFAM and the BBN. The key contributions are the followings. First, we have proposed to build a morphable partial face model, named SFAM, based on Principle Component Analysis. This model allows to learn boththe global variations in face landmark configuration and the local ones in terms of texture and local geometry around each landmark. Various partial face instances can be generated from SFAM by varying model parameters. Secondly, we have developed a landmarking algorithm based on the minimization an objective function describing the correlation between model instances and query faces. Thirdly, we have designed a Bayesian Belief Network with a structure describing the casual relationships among subjects, expressions and facial features. Facial expression oraction units are modelled as the states of the expression node and are recognized by identifying the maximum of beliefs of all states. We have also proposed a novel method for BBN parameter inference using a statistical feature model that can beconsidered as an extension of SFAM. Finally, in order to enrich information usedfor 3D face analysis, and particularly 3D facial expression recognition, we have also elaborated a 3D face feature, named SGAND, to characterize the geometry property of a point on 3D face mesh using its surrounding points.The effectiveness of all these methods has been evaluated on FRGC, BU3DFEand Bosphorus datasets for facial landmarking as well as BU3DFE and Bosphorus datasets for facial activity (expression and action unit) recognition.

Cette thèse de doctorat est dédiée à l’analyse automatique de visages 3D, incluant la détection de points d’intérêt et la reconnaissance de l’expression faciale. En effet, l’expression faciale joue un rôle important dans la communication verbale et non verbale, ainsi que pour exprimer des émotions. Ainsi, la reconnaissance automatique de l’expression faciale offre de nombreuses opportunités et applications, et est en particulier au coeur d’interfaces homme-machine "intelligentes" centrées sur l’être humain. Par ailleurs, la détection automatique de points d’intérêt du visage (coins de la bouche et des yeux, ...) permet la localisation d’éléments du visage qui est essentielle pour de nombreuses méthodes d’analyse faciale telle que la segmentation du visage et l’extraction de descripteurs utilisée par exemple pour la reconnaissance de l’expression. L’objectif de cette thèse est donc d’élaborer des approches de détection de points d’intérêt sur les visages 3D et de reconnaissance de l’expression faciale pour finalement proposer une solution entièrement automatique de reconnaissance de l’activité faciale incluant l’expression et les unités d’action (ou Action Units). Dans ce travail, nous avons proposé un réseau de croyance bayésien (Bayesian Belief Network ou BBN) pour la reconnaissance d’expressions faciales ainsi que d’unités d’action. Un modèle statistique de caractéristiques faciales (Statistical Facial feAture Model ou SFAM) a également été élaboré pour permettre la localisation des points d’intérêt sur laquelle s’appuie notre BBN afin de permettre la mise en place d’un système entièrement automatique de reconnaissance de l’expression faciale. Nos principales contributions sont les suivantes. Tout d’abord, nous avons proposé un modèle de visage partiel déformable, nommé SFAM, basé sur le principe de l’analyse en composantes principales. Ce modèle permet d’apprendre à la fois les variations globales de la position relative des points d’intérêt du visage (configuration du visage) et les variations locales en terme de texture et de forme autour de chaque point d’intérêt. Différentes instances de visages partiels peuvent ainsi être produites en faisant varier les valeurs des paramètres du modèle. Deuxièmement, nous avons développé un algorithme de localisation des points d’intérêt du visage basé sur la minimisation d’une fonction objectif décrivant la corrélation entre les instances du modèle SFAM et les visages requête. Troisièmement, nous avons élaboré un réseau de croyance bayésien (BBN) dont la structure décrit les relations de dépendance entre les sujets, les expressions et les descripteurs faciaux. Les expressions faciales et les unités d’action sont alors modélisées comme les états du noeud correspondant à la variable expression et sont reconnues en identifiant le maximum de croyance pour tous les états. Nous avons également proposé une nouvelle approche pour l’inférence des paramètres du BBN utilisant un modèle de caractéristiques faciales pouvant être considéré comme une extension de SFAM. Finalement, afin d’enrichir l’information utilisée pour l’analyse de visages 3D, et particulièrement pour la reconnaissance de l’expression faciale, nous avons également élaboré un descripteur de visages 3D, nommé SGAND, pour caractériser les propriétés géométriques d’un point par rapport à son voisinage dans le nuage de points représentant un visage 3D. L’efficacité de ces méthodes a été évaluée sur les bases FRGC, BU3DFE et Bosphorus pour la localisation des points d’intérêt ainsi que sur les bases BU3DFE et Bosphorus pour la reconnaissance des expressions faciales et des unités d’action. / This Ph.D thesis work is dedicated to automatic facial analysis in 3D, including facial landmarking and facial expression recognition. Indeed, facial expression plays an important role both in verbal and non verbal communication, and in expressing emotions. Thus, automatic facial expression recognition has various purposes and applications and particularly is at the heart of "intelligent" human-centered human/computer(robot) interfaces. Meanwhile, automatic landmarking provides aprior knowledge on location of face landmarks, which is required by many face analysis methods such as face segmentation and feature extraction used for instance for expression recognition. The purpose of this thesis is thus to elaborate 3D landmarking and facial expression recognition approaches for finally proposing an automatic facial activity (facial expression and action unit) recognition solution.In this work, we have proposed a Bayesian Belief Network (BBN) for recognizing facial activities, such as facial expressions and facial action units. A StatisticalFacial feAture Model (SFAM) has also been designed to first automatically locateface landmarks so that a fully automatic facial expression recognition system can be formed by combining the SFAM and the BBN. The key contributions are the followings. First, we have proposed to build a morphable partial face model, named SFAM, based on Principle Component Analysis. This model allows to learn boththe global variations in face landmark configuration and the local ones in terms of texture and local geometry around each landmark. Various partial face instances can be generated from SFAM by varying model parameters. Secondly, we have developed a landmarking algorithm based on the minimization an objective function describing the correlation between model instances and query faces. Thirdly, we have designed a Bayesian Belief Network with a structure describing the casual relationships among subjects, expressions and facial features. Facial expression oraction units are modelled as the states of the expression node and are recognized by identifying the maximum of beliefs of all states. We have also proposed a novel method for BBN parameter inference using a statistical feature model that can beconsidered as an extension of SFAM. Finally, in order to enrich information usedfor 3D face analysis, and particularly 3D facial expression recognition, we have also elaborated a 3D face feature, named SGAND, to characterize the geometry property of a point on 3D face mesh using its surrounding points.The effectiveness of all these methods has been evaluated on FRGC, BU3DFEand Bosphorus datasets for facial landmarking as well as BU3DFE and Bosphorus datasets for facial activity (expression and action unit) recognition.