This dissertation introduces work on face recognition using a novel technique based on Hidden Markov Models (HMMs). Through the integration of a priori structural knowledge with statistical information, HMMs can be used successfully to encode face features. The results reported are obtained using a database of images of 40 subjects, with 5 training images and 5 test images for each. It is shown how standard one-dimensional HMMs in the shape of top-bottom models can be parameterised, yielding successful recognition rates of up to around 85%. The insights gained from top-bottom models are extended to pseudo two-dimensional HMMs, which offer a better and more flexible model, that describes some of the twodimensional dependencies missed by the standard one-dimensional model. It is shown how pseudo two-dimensional HMMs can be implemented, yielding successful recognition rates of up to around 95%. The performance of the HMMs is compared with the Eigenface approach and various domain and resolution experiments are also carried out. Finally, the performance of the HMM is evaluated in a fully automated system, where database images are cropped automatically.

One of the central problems in computer vision is the automatic recognition of object classes. In particular, the detection of the class of human faces is a problem that generates special interest due to the large number of applications that require face detection as a first step. In this thesis we approach the problem of face detection as a joint detection and segmentation problem, in order to precisely localize faces with pixel accurate masks. Even though this is our primary goal, in finding a solution we have tried to create a general framework as independent as possible of the type of object being searched. For that purpose, the technique relies on a hierarchical region-based image model, the Binary Partition Tree, where objects are obtained by the union of regions in an image partition. In this work, this model is optimized for the face detection and segmentation tasks. Different merging and stopping criteria are proposed and compared through a large set of experiments. In the proposed system the intra-class variability of faces is managed within a learning framework. The face class is characterized using a set of descriptors measured on the tree nodes, and a set of one-class classifiers. The system is formed by two strong classifiers. First, a cascade of binary classifiers simplifies the search space, and afterwards, an ensemble of more complex classifiers performs the final classification of the tree nodes. The system is extensively tested on different face data sets, producing accurate segmentations and proving to be quite robust to variations in scale, position, orientation, lighting conditions and background complexity. We show that the technique proposed for faces can be easily adapted to detect other object classes. Since the construction of the image model does not depend on any object class, different objects can be detected and segmented using the appropriate object model on the same image model. New object models can be easily built by selecting and training a suitable set of descriptors and classifiers. Finally, a tracking mechanism is proposed. It combines the efficiency of the mean-shift algorithm with the use of regions to track and segment faces through a video sequence, where both the face and the camera may move. The method is extended to deal with other deformable objects, using a region-based graph-cut method for the final object segmentation at each frame. Experiments show that both mean-shift based trackers produce accurate segmentations even in difficult scenarios such as those with similar object and background colors and fast camera and object movements. Lloc i / Un dels problemes més importants en l'àrea de visió artificial és el reconeixement automàtic de classes d'objectes. En particular, la detecció de la classe de cares humanes és un problema que genera especial interès degut al gran nombre d'aplicacions que requereixen com a primer pas detectar les cares a l'escena. A aquesta tesis s'analitza el problema de detecció de cares com un problema conjunt de detecció i segmentació, per tal de localitzar de manera precisa les cares a l'escena amb màscares que arribin a precisions d'un píxel. Malgrat l'objectiu principal de la tesi és aquest, en el procés de trobar una solució s'ha intentat crear un marc de treball general i tan independent com fos possible del tipus d'objecte que s'està buscant. Amb aquest propòsit, la tècnica proposada fa ús d'un model jeràrquic d'imatge basat en regions, l'arbre binari de particions (BPT: Binary Partition Tree), en el qual els objectes s'obtenen com a unió de regions que provenen d'una partició de la imatge. En aquest treball, s'ha optimitzat el model per a les tasques de detecció i segmentació de cares. Per això, es proposen diferents criteris de fusió i de parada, els quals es comparen en un conjunt ampli d'experiments. En el sistema proposat, la variabilitat dins de la classe cara s'estudia dins d'un marc de treball d'aprenentatge automàtic. La classe cara es caracteritza fent servir un conjunt de descriptors, que es mesuren en els nodes de l'arbre, així com un conjunt de classificadors d'una única classe. El sistema està format per dos classificadors forts. Primer s'utilitza una cascada de classificadors binaris que realitzen una simplificació de l'espai de cerca i, posteriorment, s'aplica un conjunt de classificadors més complexes que produeixen la classificació final dels nodes de l'arbre. El sistema es testeja de manera exhaustiva sobre diferents bases de dades de cares, sobre les quals s'obtenen segmentacions precises provant així la robustesa del sistema en front a variacions d'escala, posició, orientació, condicions d'il·luminació i complexitat del fons de l'escena. A aquesta tesi es mostra també que la tècnica proposada per cares pot ser fàcilment adaptable a la detecció i segmentació d'altres classes d'objectes. Donat que la construcció del model d'imatge no depèn de la classe d'objecte que es pretén buscar, es pot detectar i segmentar diferents classes d'objectes fent servir, sobre el mateix model d'imatge, el model d'objecte apropiat. Nous models d'objecte poden ser fàcilment construïts mitjançant la selecció i l'entrenament d'un conjunt adient de descriptors i classificadors. Finalment, es proposa un mecanisme de seguiment. Aquest mecanisme combina l'eficiència de l'algorisme mean-shift amb l'ús de regions per fer el seguiment i segmentar les cares al llarg d'una seqüència de vídeo a la qual tant la càmera com la cara es poden moure. Aquest mètode s'estén al cas de seguiment d'altres objectes deformables, utilitzant una versió basada en regions de la tècnica de graph-cut per obtenir la segmentació final de l'objecte a cada imatge. Els experiments realitzats mostren que les dues versions del sistema de seguiment basat en l'algorisme mean-shift produeixen segmentacions acurades, fins i tot en entorns complicats com ara quan l'objecte i el fons de l'escena presenten colors similars o quan es produeix un moviment ràpid, ja sigui de la càmera o de l'objecte.

In this paper is tested method for detection skin on the driver's head. It is based on skin color and motion detection. We also tested method for eye detection in image.

In this thesis, the expression variation problem in two-dimensional (2D) and three-dimensional (3D) face recognition is tackled. While discriminant analysis (DA) methods are effective solutions for recognizing expression-variant 2D face images, they are not directly applicable when only a single sample image per subject is available. This problem is addressed in this thesis by introducing expression subspaces which can be used for synthesizing new expression images from subjects with only one sample image. It is proposed that by augmenting a generic training set with the gallery and their synthesized new expression images, and then training DA methods using this new set, the face recognition performance can be significantly improved. An important advantage of the proposed method is its simplicity; the expression of an image is transformed simply by projecting it into another subspace. The above proposed solution can also be used in general pattern recognition applications. The above method can also be used in 3D face recognition where expression variation is a more serious issue. However, DA methods cannot be readily applied to 3D faces because of the lack of a proper alignment method for 3D faces. To solve this issue, a method is proposed for sampling the points of the face that correspond to the same facial features across all faces, denoted as the closest-normal points (CNPs). It is shown that the performance of the linear discriminant analysis (LDA) method, applied to such an aligned representation of 3D faces, is significantly better than the performance of the state-of-the-art methods which, rely on one-by-one registration of the probe faces to every gallery face. Furthermore, as an important finding, it is shown that the surface normal vectors of the face provide a higher level of discriminatory information rather than the coordinates of the points. In addition, the expression subspace approach is used for the recognition of 3D faces from single sample. By constructing expression subspaces from the surface normal vectors at the CNPs, the surface normal vectors of a 3D face with single sample can be synthesized under other expressions. As a result, by improving the estimation of the within-class scatter matrix using the synthesized samples, a significant improvement in the recognition performance is achieved.

In this thesis, the expression variation problem in two-dimensional (2D) and three-dimensional (3D) face recognition is tackled. While discriminant analysis (DA) methods are effective solutions for recognizing expression-variant 2D face images, they are not directly applicable when only a single sample image per subject is available. This problem is addressed in this thesis by introducing expression subspaces which can be used for synthesizing new expression images from subjects with only one sample image. It is proposed that by augmenting a generic training set with the gallery and their synthesized new expression images, and then training DA methods using this new set, the face recognition performance can be significantly improved. An important advantage of the proposed method is its simplicity; the expression of an image is transformed simply by projecting it into another subspace. The above proposed solution can also be used in general pattern recognition applications. The above method can also be used in 3D face recognition where expression variation is a more serious issue. However, DA methods cannot be readily applied to 3D faces because of the lack of a proper alignment method for 3D faces. To solve this issue, a method is proposed for sampling the points of the face that correspond to the same facial features across all faces, denoted as the closest-normal points (CNPs). It is shown that the performance of the linear discriminant analysis (LDA) method, applied to such an aligned representation of 3D faces, is significantly better than the performance of the state-of-the-art methods which, rely on one-by-one registration of the probe faces to every gallery face. Furthermore, as an important finding, it is shown that the surface normal vectors of the face provide a higher level of discriminatory information rather than the coordinates of the points. In addition, the expression subspace approach is used for the recognition of 3D faces from single sample. By constructing expression subspaces from the surface normal vectors at the CNPs, the surface normal vectors of a 3D face with single sample can be synthesized under other expressions. As a result, by improving the estimation of the within-class scatter matrix using the synthesized samples, a significant improvement in the recognition performance is achieved.

The focus of this thesis is on facial biometrics; specifically in the problems of face detection and face recognition. Despite intensive research over the last 20 years, the technology is not foolproof, which is why we do not see use of face recognition systems in critical sectors such as banking. In this thesis, we focus on three sub-problems in these two areas of research. Firstly, we propose methods to improve the speed-accuracy trade-off of the state-of-the-art face detector. Secondly, we consider a problem that is often ignored in the literature: to decrease the training time of the detectors. We propose two techniques to this end. Thirdly, we present a detailed large-scale study on self-updating face recognition systems in an attempt to answer if continuously changing facial appearance can be learnt automatically. / L'objectiu d'aquesta tesi és sobre biometria facial, específicament en els problemes de detecció de rostres i reconeixement facial. Malgrat la intensa recerca durant els últims 20 anys, la tecnologia no és infalible, de manera que no veiem l'ús dels sistemes de reconeixement de rostres en sectors crítics com la banca. En aquesta tesi, ens centrem en tres sub-problemes en aquestes dues àrees de recerca. En primer lloc, es proposa mètodes per millorar l'equilibri entre la precisió i la velocitat del detector de cares d'última generació. En segon lloc, considerem un problema que sovint s'ignora en la literatura: disminuir el temps de formació dels detectors. Es proposen dues tècniques per a aquest fi. En tercer lloc, es presenta un estudi detallat a gran escala sobre l'auto-actualització dels sistemes de reconeixement facial en un intent de respondre si el canvi constant de l'aparença facial es pot aprendre de forma automàtica.