Automated human face recognition is a computer vision problem of considerable practical significance. Existing two dimensional (2D) face recognition techniques perform poorly for faces with uncontrolled poses, lighting and facial expressions. Face recognition technology based on three dimensional (3D) facial models is now emerging. Geometric facial models can be easily corrected for pose variations. They are illumination invariant, and provide structural information about the facial surface. Algorithms for 3D face recognition exist, however the area is far from being a matured technology. In this dissertation we address a number of open questions in the area of 3D human face recognition. Firstly, we make available to qualified researchers in the field, at no cost, a large Texas 3D Face Recognition Database, which was acquired as a part of this research work. This database contains 1149 2D and 3D images of 118 subjects. We also provide 25 manually located facial fiducial points on each face in this database. Our next contribution is the development of a completely automatic novel 3D face recognition algorithm, which employs discriminatory anthropometric distances between carefully selected local facial features. This algorithm neither uses general purpose pattern recognition approaches, nor does it directly extend 2D face recognition techniques to the 3D domain. Instead, it is based on an understanding of the structurally diverse characteristics of human faces, which we isolate from the scientific discipline of facial anthropometry. We demonstrate the effectiveness and superior performance of the proposed algorithm, relative to existing benchmark 3D face recognition algorithms. A related contribution is the development of highly accurate and reliable 2D+3D algorithms for automatically detecting 10 anthropometric facial fiducial points. While developing these algorithms, we identify unique structural/textural properties associated with the facial fiducial points. Furthermore, unlike previous algorithms for detecting facial fiducial points, we systematically evaluate our algorithms against manually located facial fiducial points on a large database of images. Our third contribution is the development of an effective algorithm for computing the structural dissimilarity of 3D facial surfaces, which uses a recently developed image similarity index called the complex-wavelet structural similarity index. This algorithm is unique in that unlike existing approaches, it does not require that the facial surfaces be finely registered before they are compared. Furthermore, it is nearly an order of magnitude more accurate than existing facial surface matching based approaches. Finally, we propose a simple method to combine the two new 3D face recognition algorithms that we developed, resulting in a 3D face recognition algorithm that is competitive with the existing state-of-the-art algorithms. / text

This Ph.D thesis work is dedicated to 3D facial surface analysis, processing as well as to the newly proposed 3D face recognition modality, which is based on mapping techniques. Facial surface processing and analysis is one of the most important steps for 3Dface recognition algorithms. Automatic anthropometric facial features localization also plays an important role for face localization, face expression recognition, face registration ect., thus its automatic localization is a crucial step for 3D face processing algorithms. In this work we focused on precise and rotation invariant landmarks localization, which are later used directly for face recognition. The landmarks are localized combining local surface properties expressed in terms of differential geometry tools and global facial generic model, used for face validation. Since curvatures, which are differential geometry properties, are sensitive to surface noise, one of the main contributions of this thesis is a modification of curvatures calculation method. The modification incorporates the surface noise into the calculation method and helps to control smoothness of the curvatures. Therefore the main facial points can be reliably and precisely localized (100% nose tip localization using 8 mm precision)under the influence of rotations and surface noise. The modification of the curvatures calculation method was also tested under different face model resolutions, resulting in stable curvature values. Finally, since curvatures analysis leads to many facial landmark candidates, the validation of which is time consuming, facial landmarks localization based on learning technique was proposed. The learning technique helps to reject incorrect landmark candidates with a high probability, thus accelerating landmarks localization. Face recognition using 3D models is a relatively new subject, which has been proposed to overcome shortcomings of 2D face recognition modality. However, 3Dface recognition algorithms are likely more complicated. Additionally, since 3D face models describe facial surface geometry, they are more sensitive to facial expression changes. Our contribution is reducing dimensionality of the input data by mapping3D facial models on to 2D domain using non-rigid, conformal mapping techniques. Having 2D images which represent facial models, all previously developed 2D face recognition algorithms can be used. In our work, conformal shape images of 3Dfacial surfaces were fed in to 2D2 PCA, achieving more than 86% recognition rate rank-one using the FRGC data set. The effectiveness of all the methods has been evaluated using the FRGC and Bosphorus datasets.

[Truncated abstract] The aim of visual recognition is to identify objects in a scene and estimate their pose. Object recognition from 2D images is sensitive to illumination, pose, clutter and occlusions. Object recognition from range data on the other hand does not suffer from these limitations. An important paradigm of recognition is model-based whereby 3D models of objects are constructed offline and saved in a database, using a suitable representation. During online recognition, a similar representation of a scene is matched with the database for recognizing objects present in the scene . . . The tensor representation is extended to automatic and pose invariant 3D face recognition. As the face is a non-rigid object, expressions can significantly change its 3D shape. Therefore, the last part of this thesis investigates representations and matching techniques for automatic 3D face recognition which are robust to facial expressions. A number of novelties are proposed in this area along with their extensive experimental validation using the largest available 3D face database. These novelties include a region-based matching algorithm for 3D face recognition, a 2D and 3D multimodal hybrid face recognition algorithm, fully automatic 3D nose ridge detection, fully automatic normalization of 3D and 2D faces, a low cost rejection classifier based on a novel Spherical Face Representation, and finally, automatic segmentation of the expression insensitive regions of a face.

Face recognition can be used in various biometric applications ranging from identifying criminals entering an airport to identifying an unconscious patient in the hospital With the introduction of 3-dimensional scanners in the last decade, researchers have begun to develop new methods for 3-D face recognition. This thesis focuses on 3-D face recognition using the one- and two-dimensional Discrete Cosine Transform (DCT) . A feature ranking based dimensionality reduction strategy is introduced to select the DCT coefficients that yield the best classification accuracies. Two forms of 3-D representation are used: point cloud and depth map images. These representations are extracted from the original VRML files in a face database and are normalized during the extraction process. Classification accuracies exceeding 97% are obtained using the point cloud images in conjunction with the 2-D DCT.

Ce travail de thèse concerne l’analyse de surfaces faciales en 3D, ainsi que leur traitement, dans le récent cadre de la modalité de reconnaissance de visages en 3D,basé sur des techniques d’appariement. Le traitement de la surface faciale et son analyse constituent une étape importante dans les algorithmes de reconnaissance de visage en 3D. La localisation de points d’intérêt anthropométriques du visage joue par ailleurs un rôle important dans les techniques de localisation du visage, de reconnaissance d’expression, de recalage, etc. Ainsi, leur localisation automatique joue un rôle crucial dans les algorithmes de traitement du visage 3D. Dans ce travail, nous avons mis l’accent sur la localisation précise et invariante en rotation des points d’intérêt, qui seront utilisés plus tard pour la reconnaissance de visages. Ces points d’intérêt sont localisés en combinant les propriétés locales de la surface faciale, exprimées en termes de géométrie différentielle, et un modèle global et générique du visage. Etant donné que la sensibilité des courbures, qui sont des propriétés de géométrie différentielle, au bruit, une des contributions de cette thèse est la modification d’une méthode de calcul de courbures. Cette modification incorpore le bruit de la surface dans la méthode de calcul, et permet de contrôler la progressivité des courbures. Par conséquent, nous pouvons localiser les points d’intérêt de la surface faciale avec précision et fiabilité (100% de bonnes localisation du bout du nez avec une erreur maximale de 8mmpar exemple) y compris en présence de rotations et de bruit. La modification de la méthode de calcul de courbure a été également testée pour différentes résolutions de visage, présentant des valeurs de courbure stables. Enfin, étant donné que donné que l’analyse de courbures mène à de nombreux candidats de points d’intérêt du visage, dont la validation est coûteuse, nous proposons de localiser les points d’intérêt grâce à une méthode d’apprentissage. Cette méthode permet de rejeter précocement des faux candidats avec une grande confiance, accélérant d’autant la localisation des points d’intérêt. La reconnaissance de visages à l’aide de modèles 3D est un sujet relativement nouveau, qui a été propose pour palier aux insuffisantes de la modalité de reconnaissance de visages en 2D. Cependant, les algorithmes de reconnaissance de visage en 3D sont généralement plus complexes. De plus, étant donné que les modèles de visage 3D décrivent la géométrie du visage, ils sont plus sensibles que les images 2Dde texture aux expressions faciales. Notre contribution est de réduire la dimensionnalité des données de départ en appariant les modèles de visage 3D au domaine 2Dà l’aide de méthodes, non rigides, d’appariement conformal. L’existence de modèles2D représentant les visages permet alors d’utiliser les techniques précédemment développées dans le domaine de la reconnaissance de visages en 2D. Dans nos travaux, nous avons utilisé les cartes conformales de visages 3D en conjonction avec l’algorithme2D2 PCA, atteignant le score de 86% en reconnaissance de rang 1 sur la base de données FRGC. L’efficacité de toutes les méthodes a été évaluée sur les bases FRGC et Bosphorus. / This Ph.D thesis work is dedicated to 3D facial surface analysis, processing as well as to the newly proposed 3D face recognition modality, which is based on mapping techniques. Facial surface processing and analysis is one of the most important steps for 3Dface recognition algorithms. Automatic anthropometric facial features localization also plays an important role for face localization, face expression recognition, face registration ect., thus its automatic localization is a crucial step for 3D face processing algorithms. In this work we focused on precise and rotation invariant landmarks localization, which are later used directly for face recognition. The landmarks are localized combining local surface properties expressed in terms of differential geometry tools and global facial generic model, used for face validation. Since curvatures, which are differential geometry properties, are sensitive to surface noise, one of the main contributions of this thesis is a modification of curvatures calculation method. The modification incorporates the surface noise into the calculation method and helps to control smoothness of the curvatures. Therefore the main facial points can be reliably and precisely localized (100% nose tip localization using 8 mm precision)under the influence of rotations and surface noise. The modification of the curvatures calculation method was also tested under different face model resolutions, resulting in stable curvature values. Finally, since curvatures analysis leads to many facial landmark candidates, the validation of which is time consuming, facial landmarks localization based on learning technique was proposed. The learning technique helps to reject incorrect landmark candidates with a high probability, thus accelerating landmarks localization. Face recognition using 3D models is a relatively new subject, which has been proposed to overcome shortcomings of 2D face recognition modality. However, 3Dface recognition algorithms are likely more complicated. Additionally, since 3D face models describe facial surface geometry, they are more sensitive to facial expression changes. Our contribution is reducing dimensionality of the input data by mapping3D facial models on to 2D domain using non-rigid, conformal mapping techniques. Having 2D images which represent facial models, all previously developed 2D face recognition algorithms can be used. In our work, conformal shape images of 3Dfacial surfaces were fed in to 2D2 PCA, achieving more than 86% recognition rate rank-one using the FRGC data set. The effectiveness of all the methods has been evaluated using the FRGC and Bosphorus datasets.

Visage est l’une des meilleures biométries pour la reconnaissance de l’identité de personnes, car l’identification d’une personne par le visage est l’habitude instinctive humaine, et l’acquisition de données faciales est naturelle, non intrusive et bien acceptée par le public. Contrairement à la reconnaissance de visage par l’image 2D sur l’apparence, la reconnaissance de visage en 3D sur la forme est théoriquement plus stable et plus robuste à la variance d’éclairage, aux petits changements de pose de la tête et aux cosmétiques pour le visage. Spécifiquement, les courbures sont les plus importants attributs géométriques pour décrire la forme géométrique d’une surface. Elles sont bénéfiques à la caractérisation de la forme du visage qui permet de diminuer l’impact des variances environnementales. Cependant, les courbures traditionnelles ne sont définies que sur des surfaces lisses. Il est donc nécessaire de généraliser telles notions sur des surfaces discrètes, par exemple des visages 3D représenté par maillage triangulaire, et d’évaluer leurs performances en reconnaissance de visage 3D. En outre, même si un certain nombre d’algorithmes 3D FR avec une grande précision sont disponibles, le coût d’acquisition de telles données de haute résolution est difficilement acceptable pour les applications pratiques. Une question majeure est donc d’exploiter les algorithmes existants pour la reconnaissance de modèles à faible résolution collecté avec l’aide d’un nombre croissant de caméras consommateur de profondeur (Kinect). Le dernier problème, mais non le moindre, est la menace sur sécurité des systèmes de reconnaissance de visage 3D par les attaques de masque fabriqué. Cette thèse est consacrée à l’étude des attributs géométriques, des mesures de courbure principale, adaptées aux maillages triangulaires, et des schémas de reconnaissance de visage 3D impliquant des telles mesures de courbure principale. En plus, nous proposons aussi un schéma de vérification sur la reconnaissance de visage 3D collecté en comparant des modèles de résolutions hétérogènes équipement aux deux résolutions, et nous évaluons la performance anti-spoofing du système de RF 3D. Finalement, nous proposons une biométrie système complémentaire de reconnaissance veineuse de main basé sur la détection de vivacité et évaluons sa performance. Dans la reconnaissance de visage 3D par la forme géométrique, nous introduisons la généralisation des courbures principales conventionnelles et des directions principales aux cas des surfaces discrètes à maillage triangulaire, et présentons les concepts des mesures de courbure principale correspondants et des vecteurs de courbure principale. Utilisant ces courbures généralisées, nous élaborons deux descriptions de visage 3D et deux schémas de reconnaissance correspondent. Avec le premier descripteur de caractéristiques, appelé Local Principal Curvature Measures Pattern (LPCMP), nous générons trois images spéciales, appelée curvature faces, correspondant à trois mesures de courbure principale et encodons les curvature faces suivant la méthode de Local Binary Pattern. Il peut décrire la surface faciale de façon exhaustive par l’information de forme locale en concaténant un ensemble d’histogrammes calculés à partir de petits patchs dans les visages de courbure. Dans le deuxième système de reconnaissance de visage 3D sans enregistrement, appelée Principal Curvature Measures based meshSIFT descriptor (PCM-meshSIFT), les mesures de courbure principales sont d’abord calculées dans l’espace de l’échelle Gaussienne, et les extrèmes de la Différence de Courbure (DoC) sont définis comme les points de caractéristique. Ensuite, nous utilisons trois mesures de courbure principales et leurs vecteurs de courbure principaux correspondants pour construire trois descripteurs locaux pour chaque point caractéristique, qui sont invariants en rotation. [...] / Face is one of the best biometrics for person recognition related application, because identifying a person by face is human instinctive habit, and facial data acquisition is natural, non-intrusive, and socially well accepted. In contrast to traditional appearance-based 2D face recognition, shape-based 3D face recognition is theoretically more stable and robust to illumination variance, small head pose changes, and facial cosmetics. The curvatures are the most important geometric attributes to describe the shape of a smooth surface. They are beneficial to facial shape characterization which makes it possible to decrease the impact of environmental variances. However, exiting curvature measurements are only defined on smooth surface. It is required to generalize such notions to discrete meshed surface, e.g., 3D face scans, and to evaluate their performance in 3D face recognition. Furthermore, even though a number of 3D FR algorithms with high accuracy are available, they all require high-resolution 3D scans whose acquisition cost is too expensive to prevent them to be implemented in real-life applications. A major question is thus how to leverage the existing 3D FR algorithms and low-resolution 3D face scans which are readily available using an increasing number of depth-consumer cameras, e.g., Kinect. The last but not least problem is the security threat from spoofing attacks on 3D face recognition system. This thesis is dedicated to study the geometric attributes, principal curvature measures, suitable to triangle meshes, and the 3D face recognition schemes involving principal curvature measures. Meanwhile, based on these approaches, we propose a heterogeneous cross-resolution 3D FR scheme, evaluate the anti-spoofing performance of shape-analysis based 3D face recognition system, and design a supplementary hand-dorsa vein recognition system based on liveness detection with discriminative power. In 3D shape-based face recognition, we introduce the generalization of the conventional point-wise principal curvatures and principal directions for fitting triangle mesh case, and present the concepts of principal curvature measures and principal curvature vectors. Based on these generalized curvatures, we design two 3D face descriptions and recognition frameworks. With the first feature description, named as Local Principal Curvature Measures Pattern descriptor (LPCMP), we generate three curvature faces corresponding to three principal curvature measures, and encode the curvature faces following Local Binary Pattern method. It can comprehensively describe the local shape information of 3D facial surface by concatenating a set of histograms calculated from small patches in the encoded curvature faces. In the second registration-free feature description, named as Principal Curvature Measures based meshSIFT descriptor (PCM-meshSIFT), the principal curvature measures are firstly computed in the Gaussian scale space, and the extremum of Difference of Curvautre (DoC) is defined as keypoints. Then we employ three principal curvature measures and their corresponding principal curvature vectors to build three rotation-invariant local 3D shape descriptors for each keypoint, and adopt the sparse representation-based classifier for keypoint matching. The comprehensive experimental results based on FRGCv2 database and Bosphorus database demonstrate that our proposed 3D face recognition scheme are effective for face recognition and robust to poses and occlusions variations. Besides, the combination of the complementary shape-based information described by three principal curvature measures significantly improves the recognition ability of system. To deal with the problem towards heterogeneous cross-resolution 3D FR, we continuous to adopt the PCM-meshSIFT based feature descriptor to perform the related 3D face recognition. [...]

Due to the natural, non-intrusive, easily collectible, widespread applicability, machine-based face recognition has received significant attention from the biometrics community over the past three decades. Compared with traditional appearance-based (2D) face recognition, shape-based (3D) face recognition is more stable to illumination variations, small head pose changes, and varying facial cosmetics. However, 3D face scans captured in unconstrained conditions may lead to various difficulties, such as non-rigid deformations caused by varying expressions, data missing due to self occlusions and external occlusions, as well as low-quality data as a result of some imperfections in the scanning technology. In order to deal with those difficulties and to be useful in real-world applications, in this thesis, we propose two 3D face recognition approaches: one is focusing on handling various expression changes, while the other one can recognize people in the presence of large facial expressions, occlusions and large pose various. In addition, we provide a provable and practical surface meshing algorithm for data-quality improvement. To deal with expression issue, we assume that different local facial region (e.g. nose, eyes) has different intra-expression/inter-expression shape variability, and thus has different importance. Based on this assumption, we design a learning strategy to find out the quantification importance of local facial regions in terms of their discriminating power. For facial description, we propose a novel shape descriptor by encoding the micro-structure of multi-channel facial normal information in multiple scales, namely, Multi-Scale and Multi-Component Local Normal Patterns (MSMC-LNP). It can comprehensively describe the local shape changes of 3D facial surfaces by a set of LNP histograms including both global and local cues. For face matching, Weighted Sparse Representation-based Classifier (W-SRC) is formulated based on the learned quantification importance and the LNP histograms. The proposed approach is evaluated on four databases: the FRGC v2.0, Bosphorus, BU-3DFE and 3D-TEC, including face scans in the presence of diverse expressions and action units, or several prototypical expressions with different intensities, or facial expression variations combine with strong facial similarities (i.e. identical twins). Extensive experimental results show that the proposed 3D face recognition approach with the use of discriminative facial descriptors can be able to deal with expression variations and perform quite accurately over all databases, and thereby has a good generalization ability. To deal with expression and data missing issues in an uniform framework, we propose a mesh-based registration free 3D face recognition approach based on a novel local facial shape descriptor and a multi-task sparse representation-based face matching process. [...]

En raison des naturelle, non-intrusive, facilement percevable caractéristiques, et une large diffusive applicabilité pour la criminalistique et de la sécurité, reconnaissance faciale basée sur la machine a reçu beaucoup d’attention de la communauté biométrie au cours des trois dernières décennies. Par rapport à la traditionnelle reconnaissance faciale basée sur le visage 2D, la reconnaissance faciale basé sur la forme 3D est plus stable aux variations d’éclairage; petite changements de tête pose, et variant cosmétiques pour le visage. Cependant, le visage 3D numérise capturé dans des conditions non-contraintes peut conduire à des difficultés diverses, comme des déformations non rigides provoquées par la variant expressions, les données manquantes en raison de l’auto-occlusion et des occlusions externes, ainsi que des données de faible qualité en raison de certaines imperfections de la technologie de numérisation. Pour régler ces difficultés et d’améliorer les applications du monde réel, dans cette thèse, nous proposons deux approches de 3D reconnaissance faciale: l’un se concentre sur le handling de divers changements d’expression, l’autre peut reconnaître les gens à la situation de présence d’un grand les expressions facial, des occlusions et des grands pose divers. En outre, nous fournissons une surface prouvable et pratique algorithme de surface maillage pour l’amélioration de la qualité de données. Pour faire face aux problème d’expression, nous supposons que la variabilité des formes de intra-expression/inter-expression de la faciale local région différent (e. g., nez, yeux) est différent, et a donc une importance niveau différente. Sur la base de cette hypothèse, nous concevons une stratégie d’apprentissage pour découvrir l’importance de la quantification de régions faciales locales en fonction de leur énergie discriminant. Pour une description du visage, nous proposons un nouveau descripteur pour coder la microstructure du multi- canal d’information normale du visage dans multiples échelles, à savoir, Multi-Scale and Multi-Component Local Normal Patterns (MSMC-LNP). On peut globalement décrire les changements de forme locale de 3D surfaces faciales par un ensemble d’histogrammes LNP y compris les indices globaux et locaux. Pour le visage correspondant, Weighted Sparse Representation-based Classifier (W-SRC) est formulée sur la base de l’importance de la quantification appris et les histogrammes LNP. L’approche proposée est évaluée sur quatre bases de données: le FRGC v2. 0, Bosphore, BU- 3DFE et 3D -TEC, y compris les scans du visage en présence de diverses expressions et des unités d’action, ou de plusieurs expressions prototypiques avec des intensités différentes, ou des variations d’expression du visage combinée avec de fortes similitudes faciales (c.à.d. jumeaux identiques). Résultats expérimentaux étendus montrent que l’approche de reconnaissance de 3D visage proposé avec l’utilisation de descripteurs discriminants du visage peut régler les variations d’expression et d’effectuer avec assez de précision sur toutes les bases de données, et a ainsi une bonne capacité de généralisation. Pour faire face à l’expression et problème des données manquantes dans un cadre uniforme, nous proposons une approche sur le sans-enregistrement maillage-basé reconnaissance du 3D visage basé sur un nouveau local descripteur de la forme du visage et un correspondance processus d’clairsemée représentation du visage basée multi- tâche. [...] / Due to the natural, non-intrusive, easily collectible, widespread applicability, machine-based face recognition has received significant attention from the biometrics community over the past three decades. Compared with traditional appearance-based (2D) face recognition, shape-based (3D) face recognition is more stable to illumination variations, small head pose changes, and varying facial cosmetics. However, 3D face scans captured in unconstrained conditions may lead to various difficulties, such as non-rigid deformations caused by varying expressions, data missing due to self occlusions and external occlusions, as well as low-quality data as a result of some imperfections in the scanning technology. In order to deal with those difficulties and to be useful in real-world applications, in this thesis, we propose two 3D face recognition approaches: one is focusing on handling various expression changes, while the other one can recognize people in the presence of large facial expressions, occlusions and large pose various. In addition, we provide a provable and practical surface meshing algorithm for data-quality improvement. To deal with expression issue, we assume that different local facial region (e.g. nose, eyes) has different intra-expression/inter-expression shape variability, and thus has different importance. Based on this assumption, we design a learning strategy to find out the quantification importance of local facial regions in terms of their discriminating power. For facial description, we propose a novel shape descriptor by encoding the micro-structure of multi-channel facial normal information in multiple scales, namely, Multi-Scale and Multi-Component Local Normal Patterns (MSMC-LNP). It can comprehensively describe the local shape changes of 3D facial surfaces by a set of LNP histograms including both global and local cues. For face matching, Weighted Sparse Representation-based Classifier (W-SRC) is formulated based on the learned quantification importance and the LNP histograms. The proposed approach is evaluated on four databases: the FRGC v2.0, Bosphorus, BU-3DFE and 3D-TEC, including face scans in the presence of diverse expressions and action units, or several prototypical expressions with different intensities, or facial expression variations combine with strong facial similarities (i.e. identical twins). Extensive experimental results show that the proposed 3D face recognition approach with the use of discriminative facial descriptors can be able to deal with expression variations and perform quite accurately over all databases, and thereby has a good generalization ability. To deal with expression and data missing issues in an uniform framework, we propose a mesh-based registration free 3D face recognition approach based on a novel local facial shape descriptor and a multi-task sparse representation-based face matching process. [...]

La reconnaissance faciale (RF) est un domaine de recherche très actif en raison de ses nombreuses applications dans le domaine de la vision par ordinateur en général et en biométrie en particulier. Cet intérêt est motivé par plusieurs raisons. D’abord, le visage est universel. Ensuite, il est le moyen le plus naturel par les êtres humains de s’identifier les uns des autres. Enfin, le visage en tant que modalité biométrique est présente un caractère non intrusif, ce qui le distingue d’autres modalités biométriques comme l’iris ou l’emprunte digitale. La RF représente aussi des défis scientifiques importants. D’abord parce que tous les visages humains ont des configurations similaires. Ensuite, avec les images faciales 2D que l’on peut acquérir facilement, la variation intra-classe, due à des facteurs comme le changement de poses et de conditions d’éclairage, les variations d’expressions faciales, le vieillissement, est bien plus importante que la variation inter-classe.Avec l’arrivée des systèmes d’acquisition 3D capables de capturer la profondeur d’objets, la reconnaissance faciale 3D (RF 3D) a émergé comme une voie prometteuse pour traiter les deux problèmes non résolus en 2D, à savoir les variations de pose et d’éclairage. En effet, les caméras 3D délivrent généralement les scans 3D de visages avec leurs images de texture alignées. Une solution en RF 3D peut donc tirer parti d’une fusion avisée d’informations de forme en 3D et celles de texture en 2D. En effet, étant donné que les scans 3D de visage offrent à la fois les surfaces faciales pour la modalité 3D pure et les images de texture 2D alignées, le nombre de possibilités de fusion pour optimiser le taux de reconnaissance est donc considérable. L’optimisation de stratégies de fusion pour une meilleure RF 3D est l’objectif principal de nos travaux de recherche menés dans cette thèse.Dans l’état d’art, diverses stratégies de fusion ont été proposées pour la reconnaissance de visages 3D, allant de la fusion précoce "early fusion" opérant au niveau de caractéristiques à la fusion tardive "late fusion" sur les sorties de classifieurs, en passant par de nombreuses stratégies intermédiaires. Pour les stratégies de fusion tardive, nous distinguons encore des combinaisons en parallèle, en cascade ou multi-niveaux. Une exploration exhaustive d’un tel espace étant impossible, il faut donc recourir à des solutions heuristiques qui constituent nos démarches de base dans le cadre des travaux de cette thèse.En plus, en s’inscrivant dans un cadre de systèmes biométriques, les critères d’optimalité des stratégies de fusion restent des questions primordiales. En effet, une stratégie de fusion est dite optimisée si elle est capable d’intégrer et de tirer parti des différentes modalités et, plus largement, des différentes informations extraites lors du processus de reconnaissance quelque soit leur niveau d’abstraction et, par conséquent, de difficulté.Pour surmonter toutes ces difficultés et proposer une solution optimisée, notre démarche s’appuie d’une part sur l’apprentissage qui permet de qualifier sur des données d’entrainement les experts 2D ou 3D, selon des critères de performance comme ERR, et d’autre part l’utilisation de stratégie d’optimisation heuristique comme le recuit simulé qui permet d’optimiser les mélanges des experts à fusionner. [...] / Face recognition (FR) was one of the motivations of computer vision for a long time, but only in recent years reliable automatic face recognition has become a realistic target of biometrics research. This interest is motivated by several reasons. First, the face is one of the most preferable biometrics for person identification and verification related applications, because it is natural, non-intrusive, and socially well accepted. The second reason relates to the challenges encountered in the FR domain, in which all human faces are similar to each other and hence offer low distinctiveness as compared with other biometrics, e.g., fingerprints and irises. Furthermore, when employing facial texture images, intra-class variations due to various factors as illumination and pose changes are usually greater than inter-class ones, preventing 2D face recognition systems from being completely reliable in real conditions.Recent, 3D acquisition systems are capable to capture the shape information of objects. Thus, 3D face recognition (3D FR) has been extensively investigated by the research community to deal with the unsolved issues in 2D face recognition, i.e., illumination and pose changes. Indeed, 3D cameras generally deliver the 3D scans of faces with their aligned texture images. 3D FR can benefit from the fusion of 2D texture and 3D shape information.This Ph.D thesis is dedicated to the optimization of fusion strategies based on three dimensional data. However, there are some problems. Indeed, since the 3D face scans provide both the facial surfaces for the 3D model and 2D texture images, the number of fusion method is high.In the literature, many fusion strategies exist that have been proposed for 3D face recognition. We can roughly classify the fusion strategies into two categories: early fusion and late fusion. Some intermediate strategies such as serial fusion and multi-level fusion have been proposed as well. Meanwhile, the search for an optimal fusion scheme remains extraordinarily complex because the cardinality of the space of possible fusion strategies. It is exponentially proportional to the number of competing features and classifiers. Thus, we require fusion technique to efficiently manage all these features and classifiers that constitute our contribution in this work. In addition, the optimality criteria of fusion strategies remain critical issues. By definition, an optimal fusion strategy is able to integrate and take advantage from different data.To overcome all these difficulties and propose an optimized solution, we adopted the following reflection. [...]

In this thesis, the expression variation problem in two-dimensional (2D) and three-dimensional (3D) face recognition is tackled. While discriminant analysis (DA) methods are effective solutions for recognizing expression-variant 2D face images, they are not directly applicable when only a single sample image per subject is available. This problem is addressed in this thesis by introducing expression subspaces which can be used for synthesizing new expression images from subjects with only one sample image. It is proposed that by augmenting a generic training set with the gallery and their synthesized new expression images, and then training DA methods using this new set, the face recognition performance can be significantly improved. An important advantage of the proposed method is its simplicity; the expression of an image is transformed simply by projecting it into another subspace. The above proposed solution can also be used in general pattern recognition applications. The above method can also be used in 3D face recognition where expression variation is a more serious issue. However, DA methods cannot be readily applied to 3D faces because of the lack of a proper alignment method for 3D faces. To solve this issue, a method is proposed for sampling the points of the face that correspond to the same facial features across all faces, denoted as the closest-normal points (CNPs). It is shown that the performance of the linear discriminant analysis (LDA) method, applied to such an aligned representation of 3D faces, is significantly better than the performance of the state-of-the-art methods which, rely on one-by-one registration of the probe faces to every gallery face. Furthermore, as an important finding, it is shown that the surface normal vectors of the face provide a higher level of discriminatory information rather than the coordinates of the points. In addition, the expression subspace approach is used for the recognition of 3D faces from single sample. By constructing expression subspaces from the surface normal vectors at the CNPs, the surface normal vectors of a 3D face with single sample can be synthesized under other expressions. As a result, by improving the estimation of the within-class scatter matrix using the synthesized samples, a significant improvement in the recognition performance is achieved.