This thesis presents a real-time multi-face tracking system, which is able to track multiple faces for live videos, broadcast, real-time conference recording, etc. The real-time output is one of the most significant advantages. Our proposed tracking system is comprised of three parts: face detection, feature extraction and tracking. We deploy a three-layer Convolutional Neural Network (CNN) to detect a face, a one-layer CNN to extract the features of a detected face and a shallow network for face tracking based on the extracted feature maps of the face. The performance of our multi-face tracking system enables the tracker to run in real-time without any on-line training. This algorithm does not need to change any parameters according to different input video conditions, and the runtime cost will not be affected significantly by an the increase in the number of faces being tracked. In addition, our proposed tracker can overcome most of the generally difficult tracking conditions which include video containing a camera cut, face occlusion, false positive face detection, false negative face detection, e.g. due to faces at the image boundary or faces shown in profile. We use two commonly used metrics to evaluate the performance of our multi-face tracking system demonstrating that our system achieves accurate results. Our multi-face tracker achieves an average runtime cost around 0.035s with GPU acceleration and this runtime cost is close to stable even if the number of tracked faces increases. All the evaluation results and comparisons are tested with four commonly used video data sets.

The focus of this project was to construct a humanoid animatronic head that had sufficient degrees of freedom to mimic human facial expression as well as human head movement and could be animated using face-tracking software to eliminate the amount of time spent on trial-and-error programming intrinsic in animatronics. As such, eight degrees of freedom were assigned to the robot: five in the face and three in the neck. From these degrees of freedom, the mechanics of the animatronic head were designed such that the neck and facial features could move with the same range and speed of a human being. Once the head was realized, various face-tracking software were utilized to analyze a pre-recorded video of a human actor and map the actors eye motion, eyebrow motion, mouth motion, and neck motion to the corresponding degrees of freedom on the robot. The corresponding values from the face-tracking software were then converted into required servomotor angles using MATLAB, which were then fed into Visual Show Automation to create a performance script that controls the motion and audio of the animatronic head during its performance.

No description available.

A biometria é a ciência que estuda a mensuração dos seres vivos. Muitos trabalhos exploram as características dos seres humanos tais como, impressão digital, íris e face, a fim de desenvolver sistemas biométricos, utilizados em diversas aplicações (monitoramento de segurança, computação ubíqua, robótica). O reconhecimento de faces é uma das técnicas biométricas mais investigadas, por ser bastante intuitiva e menos invasiva que as demais. Alguns trabalhos envolvendo essa técnica se preocupam apenas em localizar a face de um indivíduo (fazer a contagem de pessoas), enquanto outros tentam identificá-lo a partir de uma imagem. Este trabalho propõe uma abordagem capaz de identificar faces a partir de quadros de vídeo e, posteriormente, reconhecê-las por meio de técnicas de análise de imagens. Pode-se dividir o trabalho em dois módulos principais: (1) - Localização e rastreamento de faces em uma seqüência de imagens ( frames), além de separar a região rastreada da imagem; (2) - Reconhecimento de faces, identificando a qual pessoa pertence. Para a primeira etapa foi implementado um sistema de análise de movimento (baseado em subtração de quadros) que possibilitou localizar, rastrear e captar imagens da face de um indivíduo usando uma câmera de vídeo. Para a segunda etapa foram implementados os módulos de redução de informações (técnica Principal Component Analysis - PCA), de extração de características (transformada wavelet de Gabor), e o de classificação e identificação de face (distância Euclidiana e Support Vector Machine - SVM). Utilizando-se duas bases de dados de faces (FERET e uma própria - Própria), foram realizados testes para avaliar o sistema de reconhecimento implementado. Os resultados encontrados foram satisfatórios, atingindo 91,92% e 100,00% de taxa de acertos para as bases FERET e Própria, respectivamente. / Biometry is the science of measuring and analyzing biomedical data. Many works in this field have explored the characteristics of human beings, such as digital fingerprints, iris, and face to develop biometric systems, employed in various aplications (security monitoring, ubiquitous computation, robotic). Face identification and recognition are very apealing biometric techniques, as it it intuitive and less invasive than others. Many works in this field are only concerned with locating the face of an individual (for counting purposes), while others try to identify people from faces. The objective of this work is to develop a biometric system that could identify and recognize faces. The work can be divided into two major stages: (1) Locate and track in a sequence of images (frames), as well as separating the tracked region from the image; (2) Recognize a face as belonging to a certain individual. In the former, faces are captured from frames of a video camera by a motion analysis system (based on substraction of frames), capable of finding, tracking and croping faces from images of individuals. The later, consists of elements for data reductions (Principal Component Analysis - PCA), feature extraction (Gabor wavelets) and face classification (Euclidean distance and Support Vector Machine - SVM). Two faces databases have been used: FERET and a \"home-made\" one. Tests have been undertaken so as to assess the system\'s recognition capabilities. The experiments have shown that the technique exhibited a satisfactory performance, with success rates of 91.97% and 100% for the FERET and the \"home-made\" databases, respectively.

A biometria é a ciência que estuda a mensuração dos seres vivos. Muitos trabalhos exploram as características dos seres humanos tais como, impressão digital, íris e face, a fim de desenvolver sistemas biométricos, utilizados em diversas aplicações (monitoramento de segurança, computação ubíqua, robótica). O reconhecimento de faces é uma das técnicas biométricas mais investigadas, por ser bastante intuitiva e menos invasiva que as demais. Alguns trabalhos envolvendo essa técnica se preocupam apenas em localizar a face de um indivíduo (fazer a contagem de pessoas), enquanto outros tentam identificá-lo a partir de uma imagem. Este trabalho propõe uma abordagem capaz de identificar faces a partir de quadros de vídeo e, posteriormente, reconhecê-las por meio de técnicas de análise de imagens. Pode-se dividir o trabalho em dois módulos principais: (1) - Localização e rastreamento de faces em uma seqüência de imagens ( frames), além de separar a região rastreada da imagem; (2) - Reconhecimento de faces, identificando a qual pessoa pertence. Para a primeira etapa foi implementado um sistema de análise de movimento (baseado em subtração de quadros) que possibilitou localizar, rastrear e captar imagens da face de um indivíduo usando uma câmera de vídeo. Para a segunda etapa foram implementados os módulos de redução de informações (técnica Principal Component Analysis - PCA), de extração de características (transformada wavelet de Gabor), e o de classificação e identificação de face (distância Euclidiana e Support Vector Machine - SVM). Utilizando-se duas bases de dados de faces (FERET e uma própria - Própria), foram realizados testes para avaliar o sistema de reconhecimento implementado. Os resultados encontrados foram satisfatórios, atingindo 91,92% e 100,00% de taxa de acertos para as bases FERET e Própria, respectivamente. / Biometry is the science of measuring and analyzing biomedical data. Many works in this field have explored the characteristics of human beings, such as digital fingerprints, iris, and face to develop biometric systems, employed in various aplications (security monitoring, ubiquitous computation, robotic). Face identification and recognition are very apealing biometric techniques, as it it intuitive and less invasive than others. Many works in this field are only concerned with locating the face of an individual (for counting purposes), while others try to identify people from faces. The objective of this work is to develop a biometric system that could identify and recognize faces. The work can be divided into two major stages: (1) Locate and track in a sequence of images (frames), as well as separating the tracked region from the image; (2) Recognize a face as belonging to a certain individual. In the former, faces are captured from frames of a video camera by a motion analysis system (based on substraction of frames), capable of finding, tracking and croping faces from images of individuals. The later, consists of elements for data reductions (Principal Component Analysis - PCA), feature extraction (Gabor wavelets) and face classification (Euclidean distance and Support Vector Machine - SVM). Two faces databases have been used: FERET and a \"home-made\" one. Tests have been undertaken so as to assess the system\'s recognition capabilities. The experiments have shown that the technique exhibited a satisfactory performance, with success rates of 91.97% and 100% for the FERET and the \"home-made\" databases, respectively.

<p>The WinCandide application, a platform for face tracking and model based coding, had become out of date and needed to be upgraded. This report is based on the work of investigating possible open source GUIs and computer vision tool kits that could replace the old ones that are unsupported. Multi platform GUIs are of special interest.</p>

<p>This thesis aims at building a system capable of automatically extracting and parameterizing the position of a face and its features in images acquired from a low-end monocular camera. Such a challenging task is justified by the importance and variety of its possible applications, ranging from face and expression recognition to animation of virtual characters using video depicting real actors. The implementation includes the construction of Active Appearance Models of the human face from training images. The existing face model Candide-3 is used as a starting point, making the translation of the tracking parameters to standard MPEG-4 Facial Animation Parameters easy.</p><p>The Inverse Compositional Algorithm is employed to adapt the models to new images, working on a subspace where the appearance is "projected out" and thus focusing only on shape.</p><p>The algorithm is tested on a generic model, aiming at tracking different people’s faces, and on a specific model, considering one person only. In the former case, the need for improvements in the robustness of the system is highlighted. By contrast, the latter case gives good results regarding both quality and speed, with real time performance being a feasible goal for future developments.</p>

The need for real-time video analysis is rapidly increasing in today's world. The decreasing cost of powerful processors and the proliferation of affordable cameras, combined with needs for security, methods for searching the growing collection of video data, and an appetite for high-tech entertainment, have produced an environment where video processing is utilized for a wide variety of applications. Tracking is an element in many of these applications, for purposes like detecting anomalous behavior, classifying video clips, and measuring athletic performance. In this dissertation we focus on augmented reality, but the methods and conclusions are applicable to a wide variety of other areas. In particular, our work deals with achieving real-time performance while tracking with augmented reality systems using a minimum set of commercial hardware. We have built prototypes that use both existing technologies and new algorithms we have developed. While performance improvements would be possible with additional hardware, such as multiple cameras or parallel processors, we have concentrated on getting the most performance with the least equipment. Tracking is a broad research area, but an essential component of an augmented reality system. Tracking of some sort is needed to determine the location of scene augmentation. First, we investigated the effects of illumination on the pixel values recorded by a color video camera. We used the results to track a simple solid-colored object in our first augmented reality application. Our second augmented reality application tracks complex non-rigid objects, namely human faces. In the color experiment, we studied the effects of illumination on the color values recorded by a real camera. Human perception is important for many applications, but our focus is on the RGB values available to tracking algorithms. Since the lighting in most environments where video monitoring is done is close to white, (e.g., fluorescent lights in an office, incandescent lights in a home, or direct and indirect sunlight outside,) we looked at the response to "white" light sources as the intensity varied. The red, green, and blue values recorded by the camera can be converted to a number of other color spaces which have been shown to be invariant to various lighting conditions, including view angle, light angle, light intensity, or light color, using models of the physical properties of reflection. Our experiments show how well these derived quantities actually remained constant with real materials, real lights, and real cameras, while still retaining the ability to discriminate between different colors. This color experiment enabled us to find color spaces that were more invariant to changes in illumination intensity than the ones traditionally used. The first augmented reality application tracks a solid colored rectangle and replaces the rectangle with an image, so it appears that the subject is holding a picture instead. Tracking this simple shape is both easy and hard; easy because of the single color and the shape that can be represented by four points or four lines, and hard because there are fewer features available and the color is affected by illumination changes. Many algorithms for tracking fixed shapes do not run in real time or require rich feature sets. We have created a tracking method for simple solid colored objects that uses color and edge information and is fast enough for real-time operation. We also demonstrate a fast deinterlacing method to avoid "tearing" of fast moving edges when recorded by an interlaced camera, and optimization techniques that usually achieved a speedup of about 10 from an implementation that already used optimized image processing library routines. Human faces are complex objects that differ between individuals and undergo non-rigid transformations. Our second augmented reality application detects faces, determines their initial pose, and then tracks changes in real time. The results are displayed as virtual objects overlaid on the real video image. We used existing algorithms for motion detection and face detection. We present a novel method for determining the initial face pose in real time using symmetry. Our face tracking uses existing point tracking methods as well as extensions to Active Appearance Models (AAMs). We also give a new method for integrating detection and tracking data and leveraging the temporal coherence in video data to mitigate the false positive detections. While many face tracking applications assume exactly one face is in the image, our techniques can handle any number of faces. The color experiment along with the two augmented reality applications provide improvements in understanding the effects of illumination intensity changes on recorded colors, as well as better real-time methods for detection and tracking of solid shapes and human faces for augmented reality. These techniques can be applied to other real-time video analysis tasks, such as surveillance and video analysis.

One of the central problems in computer vision is the automatic recognition of object classes. In particular, the detection of the class of human faces is a problem that generates special interest due to the large number of applications that require face detection as a first step. In this thesis we approach the problem of face detection as a joint detection and segmentation problem, in order to precisely localize faces with pixel accurate masks. Even though this is our primary goal, in finding a solution we have tried to create a general framework as independent as possible of the type of object being searched. For that purpose, the technique relies on a hierarchical region-based image model, the Binary Partition Tree, where objects are obtained by the union of regions in an image partition. In this work, this model is optimized for the face detection and segmentation tasks. Different merging and stopping criteria are proposed and compared through a large set of experiments. In the proposed system the intra-class variability of faces is managed within a learning framework. The face class is characterized using a set of descriptors measured on the tree nodes, and a set of one-class classifiers. The system is formed by two strong classifiers. First, a cascade of binary classifiers simplifies the search space, and afterwards, an ensemble of more complex classifiers performs the final classification of the tree nodes. The system is extensively tested on different face data sets, producing accurate segmentations and proving to be quite robust to variations in scale, position, orientation, lighting conditions and background complexity. We show that the technique proposed for faces can be easily adapted to detect other object classes. Since the construction of the image model does not depend on any object class, different objects can be detected and segmented using the appropriate object model on the same image model. New object models can be easily built by selecting and training a suitable set of descriptors and classifiers. Finally, a tracking mechanism is proposed. It combines the efficiency of the mean-shift algorithm with the use of regions to track and segment faces through a video sequence, where both the face and the camera may move. The method is extended to deal with other deformable objects, using a region-based graph-cut method for the final object segmentation at each frame. Experiments show that both mean-shift based trackers produce accurate segmentations even in difficult scenarios such as those with similar object and background colors and fast camera and object movements. Lloc i / Un dels problemes més importants en l'àrea de visió artificial és el reconeixement automàtic de classes d'objectes. En particular, la detecció de la classe de cares humanes és un problema que genera especial interès degut al gran nombre d'aplicacions que requereixen com a primer pas detectar les cares a l'escena. A aquesta tesis s'analitza el problema de detecció de cares com un problema conjunt de detecció i segmentació, per tal de localitzar de manera precisa les cares a l'escena amb màscares que arribin a precisions d'un píxel. Malgrat l'objectiu principal de la tesi és aquest, en el procés de trobar una solució s'ha intentat crear un marc de treball general i tan independent com fos possible del tipus d'objecte que s'està buscant. Amb aquest propòsit, la tècnica proposada fa ús d'un model jeràrquic d'imatge basat en regions, l'arbre binari de particions (BPT: Binary Partition Tree), en el qual els objectes s'obtenen com a unió de regions que provenen d'una partició de la imatge. En aquest treball, s'ha optimitzat el model per a les tasques de detecció i segmentació de cares. Per això, es proposen diferents criteris de fusió i de parada, els quals es comparen en un conjunt ampli d'experiments. En el sistema proposat, la variabilitat dins de la classe cara s'estudia dins d'un marc de treball d'aprenentatge automàtic. La classe cara es caracteritza fent servir un conjunt de descriptors, que es mesuren en els nodes de l'arbre, així com un conjunt de classificadors d'una única classe. El sistema està format per dos classificadors forts. Primer s'utilitza una cascada de classificadors binaris que realitzen una simplificació de l'espai de cerca i, posteriorment, s'aplica un conjunt de classificadors més complexes que produeixen la classificació final dels nodes de l'arbre. El sistema es testeja de manera exhaustiva sobre diferents bases de dades de cares, sobre les quals s'obtenen segmentacions precises provant així la robustesa del sistema en front a variacions d'escala, posició, orientació, condicions d'il·luminació i complexitat del fons de l'escena. A aquesta tesi es mostra també que la tècnica proposada per cares pot ser fàcilment adaptable a la detecció i segmentació d'altres classes d'objectes. Donat que la construcció del model d'imatge no depèn de la classe d'objecte que es pretén buscar, es pot detectar i segmentar diferents classes d'objectes fent servir, sobre el mateix model d'imatge, el model d'objecte apropiat. Nous models d'objecte poden ser fàcilment construïts mitjançant la selecció i l'entrenament d'un conjunt adient de descriptors i classificadors. Finalment, es proposa un mecanisme de seguiment. Aquest mecanisme combina l'eficiència de l'algorisme mean-shift amb l'ús de regions per fer el seguiment i segmentar les cares al llarg d'una seqüència de vídeo a la qual tant la càmera com la cara es poden moure. Aquest mètode s'estén al cas de seguiment d'altres objectes deformables, utilitzant una versió basada en regions de la tècnica de graph-cut per obtenir la segmentació final de l'objecte a cada imatge. Els experiments realitzats mostren que les dues versions del sistema de seguiment basat en l'algorisme mean-shift produeixen segmentacions acurades, fins i tot en entorns complicats com ara quan l'objecte i el fons de l'escena presenten colors similars o quan es produeix un moviment ràpid, ja sigui de la càmera o de l'objecte.

This thesis aims at building a system capable of automatically extracting and parameterizing the position of a face and its features in images acquired from a low-end monocular camera. Such a challenging task is justified by the importance and variety of its possible applications, ranging from face and expression recognition to animation of virtual characters using video depicting real actors. The implementation includes the construction of Active Appearance Models of the human face from training images. The existing face model Candide-3 is used as a starting point, making the translation of the tracking parameters to standard MPEG-4 Facial Animation Parameters easy. The Inverse Compositional Algorithm is employed to adapt the models to new images, working on a subspace where the appearance is "projected out" and thus focusing only on shape. The algorithm is tested on a generic model, aiming at tracking different people’s faces, and on a specific model, considering one person only. In the former case, the need for improvements in the robustness of the system is highlighted. By contrast, the latter case gives good results regarding both quality and speed, with real time performance being a feasible goal for future developments.