Many real-world face recognition tasks are under unconstrained conditions such as off-angle pose variations, illumination variations, facial occlusion, facial expression, etc. In this work, we are focusing on the real-world scenarios where only the periocular region of a face is visible such as in the ISIS case. In Part I of the dissertation, we will showcase the face recognition capability based on the periocular region, which we call the periocular face recognition. We will demonstrate that face matching using the periocular region directly is more robust than the full face in terms of age-tolerant face recognition, expression-tolerant face recognition, pose-tolerant face recognition, as well as contains more cues for determining the gender information of a subject. In this dissertation, we will study direct periocular matching more comprehensively and systematically using both shallow and deep learning methods. Based on this, in Part II and Part III of the dissertation, we will continue to explore an indirect way of carrying out the periocular face recognition: periocular-based full face hallucination, because we want to capitalize on the powerful commercial face matchers and deep learning-based face recognition engines which are all trained on large-scale full face images. The reproducibility and feasibility of re-training for a proprietary facial region, such as the periocular region, is relatively low, due to the nonopen source nature of commercial face matchers as well as the amount of training data and computation power required by the deep learning based models. We will carry out the periocular-based full face hallucination based on two proposed reconstructive dictionary learning methods, including the dimensionally weighted K-SVD (DW-KSVD) dictionary learning approach and its kernel feature space counterpart using Fastfood kernel expansion approximation to reconstruct high-fidelity full face images from the periocular region, as well as two proposed generative deep learning approaches that build upon deep convolutional generative adversarial networks (DCGAN) to generate the full face from the periocular region observations, including the Gang of GANs (GoGAN) method and the discriminant nonlinear many-to-one generative adversarial networks (DNMM-GAN) for applications such as the generative open-set landmark-free frontalization (Golf) for faces and universal face optimization (UFO), which tackles an even broader set of problems than periocular based full face hallucination. Throughout Parts I-III, we will study how to handle challenging realworld scenarios such as unconstrained pose variations, unconstrained illumination conditions, and unconstrained low resolution of the periocular and facial images. Together, we aim to achieve unconstrained periocular face recognition through both direct periocular face matching and indirect periocular-based full face hallucination. In the final Part IV of the dissertation, we will go beyond and explore several new methods in deep learning that are statistically efficient for generalpurpose image recognition. Methods include the local binary convolutional neural networks (LBCNN), the perturbative neural networks (PNN), and the polynomial convolutional neural networks (PolyCNN).

A caricatura, uma imagem da face baseada no exagero de suas características peculiares, geralmente é reconhecida tão bem quanto a fotografia da face sem distorções. Para confecção das caricaturas, exageram-se as diferenças entre a imagem original e um protótipo (face média de um grupo de pessoas); e para confecção das anti-caricaturas essas diferenças são atenuadas. O objetivo desta pesquisa foi investigar se existe um grau de exagero ótimo para que a caricatura represente a face melhor que a fotografia original. Além disso, investigou- se o papel da percepção holística versus percepção componencial no processo de reconhecimento de faces. Foram geradas seis faces prototípicas, masculinas e femininas, de pessoas da população da região de Ribeirão Preto que se auto-declaram branca, parda e preta. A partir das faces prototípicas, foram gerados dois tipos de caricaturas e anticaricaturas: 1. holística: em que todas as diferenças entre a face original e a prototípica foram manipuladas, 2. parcial: em que somente as diferenças de alguns elementos faciais isolados ou combinados entre a face original e a prototípica foram manipuladas. No Experimento I os estímulos teste foram as caricaturas e anti-caricaturas holísticas. No Experimento II os estímulos foram as caricaturas e anti-caricaturas parciais. Em ambos experimentos as caricaturas e anti-caricaturas foram submetidas a julgamentos de similaridade com a face original previamente memorizada. Os resultados do Experimento I indicaram que a melhor representação da face é a fotografia sem distorção e que, nos casos em que a face é atípica em relação ao protótipo, as caricaturas tendem a ser representações tão fidedignas quanto as fotografias sem distorção. Os resultados do Experimento II apontam para a importância dos elementos peculiares no reconhecimento de faces. Comparando-se os resultados dos Experimentos I e II pode-se afirmar que o processamento de faces se dá predominantemente de forma holística e que a manipulação de elementos peculiares da face reduz mais a similaridade entre a face original e a caricatura (ou anti-caricatura) que a manipulação de elementos não-peculiares. / A caricature is an exaggeration of distinctive facial features and is generally recognized just as well as an undistorted photograph of a face. Caricatures can be generated by exaggerating the differences between a face and a prototypical face (average face) and an anticaricature can be generated by reducing those differences. The aim of this study was to investigate whether there is a degree of caricaturing that best captures facial likeness. Moreover, we investigated the role of holistic perception versus componential perception in the facial recognition process. Six prototypical faces, three male and three female, were generated by morphing photographs of Brazilian people from the region of Ribeirão Preto-SP of different races: black, white and mixed race. Two types of caricatures and anticaricatures were generated: 1, holistic: by manipulating of all the differences between a face and the prototypical faces; 2, partial: by manipulating the differences of isolated or combined features between a face and the prototypical face. The stimuli used in Experiment 1 were the holistic caricatures and anticaricatures. In Experiment 2 the stimuli were the partial caricatures and anticaricatures. In both experiments, subjects were asked to rate the similarity between the caricatures and the anticaricatures and a face previously memorized. The results of Experiment 1 provide evidence that the best representation of the face is a photograph without distortion and that, when the face is atypical, the caricatures seem to be as good as photographs without distortion. The results of Experiment 2 point to the importance of the role of distinctive features in face recognition. Comparing the results of Experiments 1 and 2, we can say that the facial recognition process is predominantly holistic but that the manipulation of distinctive facial elements reduces the similarity judgment more than the manipulation of non-distinctive features.

M. Comm. / Malicious acts such as fraud and terrorisms are continually becoming a more pressing threat. The need is growing daily for a cheap, non-intrusive technology, that does not make use of specialized equipment, which can identify individuals with or without their knowledge or permission, over the internet or in the public domain. The answer to this problem might be digital facial recognition, the authentication of a person according to the measurements and shape of his facial patterns (nodal points). Thus far the technology has primarily been used by law enforcement. The great strength of facial recognition is that it can scan multiple people in an area quickly, with or without their interaction with the system. The purpose of facial recognition surveillance is to implement it anywhere possible, for example shopping centres, street corners, hotel lobbies or train stations, and to be able to identify any individual finding himself in any of these areas. Also, if a larger system can be implemented, we would be able to track any individual wherever he goes. Through this, any suspicious character can be monitored and tracked if the need arises, ensuring that people can live in a much safer world.

Abstract Automatic face recognition is a challenging problem which has received much attention during recent years due to its many applications in different fields such as law enforcement, security applications, human-machine interaction etc. Up to date there is no technique that provides a robust solution for all situations and different applications. From still gray images to face sequences (and passing through color images), this thesis provides new algorithms to learn, detect and recognize faces. It also analyzes some emerging directions such as the integration of facial dynamics in the recognition process. To recognize faces, the thesis proposes a new approach based on Local Binary Patterns (LBP) which consists of dividing the facial image into small regions from which LBP features are extracted and concatenated into a single feature histogram efficiently representing the face image. Then, face recognition is performed using a nearest neighbor classifier in the computed feature space with Chi-square as a dissimilarity metric. The extensive experiments clearly show the superiority of the proposed method over the state-of the-art algorithms on FERET tests. To detect faces, another LBP-based representation which is suitable for low-resolution images, is derived. Using the new representation, a second-degree polynomial kernel SVM classifier is trained to detect frontal faces in complex gray scale images. Experimental results using several complex images show that the proposed approach performs favorably compared to the state-of-art methods. Additionally, experiments with detecting and recognizing low-resolution faces are carried out to demonstrate that the same facial representation can be efficiently used for both the detection and recognition of faces in low-resolution images. To detect faces when the color cue is available, the thesis proposes an approach based on a robust model of skin color, called a skin locus, which is used to extract the skin-like regions. After orientation normalization and based on verifying a set of criteria (face symmetry, presence of some facial features, variance of pixel intensities and connected component arrangement), only facial regions are selected. To learn and visualize faces in video sequences, the recently proposed algorithms for unsupervised learning and dimensionality reduction (LLE and ISOMAP), as well as well known ones (PCA, SOM etc.) are considered and investigated. Some extensions are proposed and a new approach for selecting face models from video sequences is developed. The approach is based on representing the face manifold in a low-dimensional space using the Locally Linear Embedding (LLE) algorithm and then performing K-means clustering. To analyze the emerging direction in face recognition which consists of combining facial shape and dynamic personal characteristics for enhancing face recognition performance, the thesis considers two factors (face sequence length and image quality) and studies their effects on the performance of video-based systems which attempt to use a spatio-temporal representation instead of a still image based one. The extensive experimental results show that motion information enhances automatic recognition but not in a systematic way as in the human visual system. Finally, some key findings of the thesis are considered and used for building a system for access control based on detecting and recognizing faces.

Abstract Image appearance descriptors are needed for different computer vision applications dealing with, for example, detection, recognition and classification of objects, textures, humans, etc. Typically, such descriptors should be discriminative to allow for making the distinction between different classes, yet still robust to intra-class variations due to imaging conditions, natural changes in appearance, noise, and other factors. The purpose of this thesis is the development and analysis of photometric descriptors for the appearance of real life images. The two application areas included in this thesis are face recognition and texture classification. To facilitate the development and analysis of descriptors, a general framework for image description using statistics of quantized filter bank responses modeling their joint distribution is introduced. Several texture and other image appearance descriptors, including the local binary pattern operator, can be presented using this model. This framework, within which the thesis is presented, enables experimental evaluation of the significance of each of the components of this three-part chain forming a descriptor from an input image. The main contribution of this thesis is a face representation method using distributions of local binary patterns computed in local rectangular regions. An important factor of this contribution is to view feature extraction from a face image as a texture description problem. This representation is further developed into a more precise model by estimating local distributions based on kernel density estimation. Furthermore, a face recognition method tolerant to image blur using local phase quantization is presented. The thesis presents three new approaches and extensions to texture analysis using quantized filter bank responses. The first two aim at increasing the robustness of the quantization process. The soft local binary pattern operator accomplishes this by making a soft quantization to several labels, whereas Bayesian local binary patterns make use of a prior distribution of labelings, and aim for the one maximizing the a posteriori probability. Third, a novel method for computing rotation invariant statistics from histograms of local binary pattern labels using the discrete Fourier transform is introduced. All the presented methods have been experimentally validated using publicly available image datasets and the results of experiments are presented in the thesis. The face description approach proposed in this thesis has been validated in several external studies, and it has been utilized and further developed by several research groups working on face analysis.

No description available.

This dissertation establishes a novel system for human face learning and recognition based on incremental multilinear Principal Component Analysis (PCA). Most of the existing face recognition systems need training data during the learning process. The system as proposed in this dissertation utilizes an unsupervised or weakly supervised learning approach, in which the learning phase requires a minimal amount of training data. It also overcomes the inability of traditional systems to adapt to the testing phase as the decision process for the newly acquired images continues to rely on that same old training data set. Consequently when a new training set is to be used, the traditional approach will require that the entire eigensystem will have to be generated again. However, as a means to speed up this computational process, the proposed method uses the eigensystem generated from the old training set together with the new images to generate more effectively the new eigensystem in a so-called incremental learning process. In the empirical evaluation phase, there are two key factors that are essential in evaluating the performance of the proposed method: (1) recognition accuracy and (2) computational complexity. In order to establish the most suitable algorithm for this research, a comparative analysis of the best performing methods has been carried out first. The results of the comparative analysis advocated for the initial utilization of the multilinear PCA in our research. As for the consideration of the issue of computational complexity for the subspace update procedure, a novel incremental algorithm, which combines the traditional sequential Karhunen-Loeve (SKL) algorithm with the newly developed incremental modified fast PCA algorithm, was established. In order to utilize the multilinear PCA in the incremental process, a new unfolding method was developed to affix the newly added data at the end of the previous data. The results of the incremental process based on these two methods were obtained to bear out these new theoretical improvements. Some object tracking results using video images are also provided as another challenging task to prove the soundness of this incremental multilinear learning method.

With more and more images being uploaded to social networks each day, the resources for identifying a large portion of the world are available. However the tools to harness and utilize this information are not sufficient. This thesis presents a system, called PhacePhinder, which can build a face database from a social network and have it accessible from mobile devices. Through combining existing technologies, this is made possible. It also makes use of a fusion probabilistic latent semantic analysis to determine strong connections between users and content. Using this information we can determine the most meaningful social connection to a recognized person, allowing us to inform the user of how they know the person being recognized. We conduct a series of offline and user tests to verify our results and compare them to existing algorithms. We show, that through combining a user’s friendship information as well as picture occurrence information, we can make stronger recommendations than based on friendship alone. We demonstrate a working prototype that can identify a face from a picture taken from a mobile phone, using a database derived from images gathered directly from a social network, and return a meaningful social connection to the recognized face.

No description available.

The own-group bias in face recognition (OGB) is the greater facility to distinguish and recognize people from one's own group at the expense of people from other-groups. The OGB has been studied for many years, however, very little research focuses on finding a way to decrease or eliminate it, through training. Reporting five studies involving memory or matching tasks, the aim of the present thesis was to develop and to explore to what extent training can decrease or remove the OGB. French White participants, and South African White, Black and Coloured participants took part in different studies, using Black and White faces as stimuli. In each study, White participants from both countries presented the expected OGB prior to any intervention. However, the presence of the OGB in South African Black participants was detected only in one (matching task) study, instead recording a higher discrimination performance by Black participants for White faces in the other studies. As expected, South African Coloured participants did not display increased discrimination performance for any of the other stimuli groups, both being out-group stimuli. Results from the training studies revealed either (a) no effect of a distributed training in feature focus over 5 weeks; (b) an increase of the OGB after a focus on critical facial features; (c) a decrease of the OGB in a task-specific training using pictures whose quality had been manipulated, and; (d) an important implication of the presence/absence of the target in a field detection study. With some promising results, the present work contributes to our understanding of how training could be used to improve face-recognition, and especially other-group face recognition.