Die Datenbestände lokaler Fernsehsender umfassen oftmals mehrere zehntausend Videokassetten. Moderne Verfahren werden benötigt, um derartige Datenkollektionen inhaltlich automatisiert zu erschließen. Das Auffinden relevanter Objekte spielt dabei eine übergeordnete Rolle, wobei gesteigerte Anforderungen wie niedrige Fehler- und hohe Detektionsraten notwendig sind, um eine Korruption des Suchindex zu verhindern und erfolgreiche Recherchen zu ermöglichen. Zugleich müssen genügend Objekte indiziert werden, um Aussagen über den tatsächlichen Inhalt zu treffen. Diese Arbeit befasst sich mit der Anpassung und Optimierung bestehender Detektionsverfahren. Dazu wird ein auf die hohen Leistungsbedürfnisse der Videoanalyse zugeschnittenes holistisches Workflow- und Prozesssystem mit der Zielstellung implementiert, die Entwicklung von Bilderkennungsalgorithmen, die Visualisierung von Zwischenschritten sowie deren Evaluation zu ermöglichen. Im Fokus stehen Verfahren zur strukturellen Zerlegung von Videomaterialien und zur inhaltlichen Analyse im Bereich der Gesichtsdetektion und Fußgängererkennung. / The data collections of local television stations often consist of multiples of ten thousand video tapes. Modern methods are needed to exploit the content of such archives. While the retrieval of objects plays a fundamental role, essential requirements incorporate low false and high detection rates in order to prevent the corruption of the search index. However, a sufficient number of objects need to be found to make assumptions about the content explored. This work focuses on the adjustment and optimization of existing detection techniques. Therefor, the author develops a holistic framework that directly reflects on the high demands of video analysis with the aim to facilitate the development of image processing algorithms, the visualization of intermediate results, and their evaluation and optimization. The effectiveness of the system is demonstrated on the structural decomposition of video footage and on content-based detection of faces and pedestrians.

在電腦視覺的研究之中，有關物件的偵測與追蹤應用在速度及可靠性上的追求一直是相當具有挑戰性的問題，而現階段發展以視覺為基礎互動式的應用，所使用到技術諸如:類神經網路、SVM及貝氏網路等。 本論文中我們持續深入此領域，並提出及發展一個方向性邊緣特徵集(DEM)與修正後的AdaBoost訓練演算法相互結合，期能有效提高物件偵測與識別的速度及準確性，在實際驗證中，我們將之應用於多種角度之人臉偵測，以及臉部表情識別等兩個主要問題之上；在人臉偵測的應用中，我們使用CMU的臉部資料庫並與Viola & Jones方法進行分析比較，在準確率上，我們的方法擁有79% 的recall及90% 的precision，而Viola & ones的方法則分別為81%及77%；在運算速度上，同樣處理512x384的影像，相較於Viola & Jones需時132ms，我們提出的方法則有較佳的82ms。 此外，於表情識別的應用中，我們結合運用Component-based及Action-unit model 兩種方法。前者的優勢在於提供臉部細節特徵的定位及追蹤變化，後者主要功用則為進行情緒表情的分類。我們對於四種不同情緒表情的辨識準確度如下：高興(83.6%)、傷心(72.7%)、驚訝(80%) 、生氣(78.1%)。在實驗中，可以發現生氣及傷心兩種情緒較難區分，而高興與驚訝則較易識別。 / Rapid and robust detection and tracking of objects is a challenging problem in computer vision research. Techniques such as artificial neural networks, support vector machine and Bayesian networks have been developed to enable interactive vision-based applications. In this thesis, we tackle this issue by devising a novel feature descriptor named directional edge maps (DEM). When combined with a modified AdaBoost training algorithm, the proposed descriptor can produce effective results in many object detection and recognition tasks. We have applied the newly developed method to two important object recognition problems, namely, face detection and facial expression recognition. The DEM-based methodology conceived in this thesis is capable of detecting faces of multiple views. To test the efficacy of our face detection mechanism, we have performed a comparative analysis with the Viola and Jones algorithm using Carnegie Mellon University face database. The recall and precision using our approach is 79% and 90%, respectively, compared to 81% and 77% using Viola and Jones algorithm. Our algorithm is also more efficient, requiring only 82 ms (compared to 132 ms by Viola and Jones) for processing a 512x384 image. To achieve robust facial expression recognition, we have combined component-based methods and action-unit model-based approaches. The component-based method is mainly utilized to locate important facial features and track their deformations. Action-unit model-based approach is then employed to carry out expression recognition. The accuracy of classifying different emotion type is as follows: happiness 83.6%, sadness 72.7%, surprise 80%, and anger 78.1%. It turns out that anger and sadness are more difficult to distinguish, whereas happiness and surprise expression have higher recognition rates.

O crescimento da utilização de sistemas de reconhecimento no mundo contemporâneo exige processos de detecção cada vez mais robustos e ágeis. Aplicáveis desde sistemas de teleconferência empresarial até mecanismos de segurança e vigilância, a detecção e o reconhecimento de pessoas tornaram-se uma constante. Na tentativa de buscar caminhos alternativos, tanto para os problemas de detecção, quanto para os de reconhecimento, este trabalho propõe a utilização de medidas biométricas, mensuradas em imagens digitalizadas de faces humanas. A partir do estudo de tais medidas, torna-se possível a verificação de proporções existentes na face, especialmente a proporção divina, podendo constituir, no futuro, a base para algoritmos de detecção e/ou reconhecimento que usufruam das informações trazidas por tais proporções. Diante de uma reduzida quantidade de publicações no meio científico que utilizam a proporção divina como meio de detecção e/ou reconhecimento em processamento de imagens, esta investigação vem contribuir com alguns passos nessa direção / The increase of recognition systems in the contemporary world has demanded robust and agile detection processes. From teleconference systems to security and monitoring mechanisms, the detection and recognition of people have became constantly used and applied. In attempt to search for alternative ways to solve both detection and recognition problems, this work proposes the utilization of biometric measures, taken in digital image of human faces. From the study of such measures, it’s possible to verify face proportions, especially the divine proportion, which could allows, in the future, to implement the detection and/or recognition algorithms that utilize such proportions. Due to small amount of scientific publications that use the divine proportion as a way of detection and/or recognition in image processing, this investigation contributes with some steps in this direction

Dans cette thèse, nous explorons l'utilisation des dérivées Gaussiennes multi-échelles comme représentation initiale pour la détection, la reconnaissance et la classification des visages humains dans des images. Nous montrons qu'un algorithme rapide, $O(N)$, de construction d'une pyramide binomiale peut être utilisé pour extraire des dérivées Gaussiennes avec une réponse impulsionnelle identique à un facteur d'échelle $sqrt{2}$>. Nous montrons ensuite qu'un vecteur composé de ces dérivées à différentes échelles et à différents ordres en chaque pixel peut être utilisé comme base pour les algorithmes de détection, de classification et de reconnaissance lesquels atteignent ou dépassent les performances de l'état de l'art avec un coût de calcul réduit. De plus l'utilisation de coefficients entiers, avec une complexité de calcul et des exigences mémoires en $O(N)$ font qu'une telle approche est appropriée pour des applications temps réel embarquées sur des systèmes mobiles. Nous testons cette représentation en utilisant trois problèmes classiques d'analyse d'images faciales : détection de visages, reconnaissance de visages et estimation de l'âge. Pour la détection de visages, nous examinons les dérivées Gaussiennes multi-échelles comme une alternative aux ondelettes de Haar pour une utilisation dans la construction d'une cascade de classifieurs linéaires appris avec l'algorithme Adaboost, popularisé par Viola and Jones. Nous montrons que la représentation pyramidale peut être utilisée pour optimiser le processus de détection en adaptant la position des dérivées dans la cascade. Dans ces experiences nous sommes capables de montrer que nous pouvons obtenir des niveaux de performances de détection similaires (mesurés par des courbes ROC) avec une réduction importante du coût de calcul. Pour la reconnaissance de visages et l'estimation de l'âge, nous montrons que les dérivées Gaussiennes multi-échelles peuvent être utilisées pour calculer une représentation tensorielle qui conserve l'information faciale la plus importante. Nous montrons que combinée à l'Analyse Multilinéaire en Composantes Principales et à la méthode Kernel Discriminative Common Vectors (KDCV), cette représentation tensorielle peut mener à un algorithme qui est similaire aux techniques concurrentes pour la reconnaissance de visages avec un coût de calcul réduit. Pour l'estimation de l'âge à partir d'images faciales, nous montrons que notre représentation tensorielle utilisant les dérivées de Gaussiennes multi-échelles peut être utilisée avec une machine à vecteur de pertinence pour fournir une estimation de l'âge avec des niveaux de performances similaires aux méthodes de l'état de l'art. / In this thesis, we propose to modelize facial images using Gaussian Derivatives computed with a Half-Octave Gaussian Pyramid. In this scope, Gaussian derivatives have shown a high versatility in object recognition and image analysis, nevertheless there is not a considerable number of proposed aproaches in the state-of-the-art that uses Gaussian derivatives for extracting important information from facial images. Motivated by the above mentioned and the high amount of applications in facial analysis, security systems and Biometry, in this thesis as a first time, we propose to use an unique image representation, the Gaussian Scale Space computed with a half octave pyramid. We show in this thesis that this image representation could be used to perform different tasks in facial analysis without lost of performance compared with other approaches in the state-of-the-art that uses more complicated image representations. it is also well know that using an unique image represenation could be convenient in real world applications where the amount of memory capacity is limitated by hardware constraints. To demostrate our assumptations we solve three different tasks in facial analysis: Face detection, Face recognition and Age estimation. In face detection we propose to use a cascade of classifiers using Gaussian derivatives. Specifically we propose to use Gaussian derivatives up to the fourth order, in effect experiemnts using different derivatives orders have shown that fourth order Gaussian derivatives provide important information in face detection and recognition. In adition, to improve the speed of detection using Gaussian derivatives, we develope a new cascade architecture which considerates the computational cost of each Gaussian derivative order to chose its best position in the cascade. Finally, to solve the face recognition and age estimation problems, we propose a tensorial model based in Gaussian derivatives. This tensorial model preserves the 3-D structure of feature space and it does not break the natural structure of data when a vectorization process is applied. Each one of the methods proposed in the thesis are discused and validated with a set of well defined experiments. All our results are compared with the last state-of-the-art results in face detection, recognition and age estimation, giving comparable or superior results

Emotion detection, also known as Facial expression recognition, is the art of mapping an emotion to some sort of input data taken from a human. This is a powerful tool to extract valuable information from individuals which can be used as data for many different purposes, ranging from medical conditions such as depression to customer feedback. To be able to solve the problem of facial expression recognition, smaller subtasks are required and all of them together form the complete system to the problem. Breaking down the bigger task at hand, one can think of these smaller subtasks in the form of a pipeline that implements the necessary steps for classification of some input to then give an output in the form of emotion. In recent time with the rise of the art of computer vision, images are often used as input for these systems and have shown great promise to assist in the task of facial expression recognition as the human face conveys the subjects emotional state and contain more information than other inputs, such as text or audio. Many of the current state-of-the-art systems utilize computer vision in combination with another rising field, namely AI, or more specifically deep learning. These proposed methods for deep learning are in many cases using a special form of neural network called convolutional neural network that specializes in extracting information from images. Then performing classification using the SoftMax function, acting as the last part before the output in the facial expression pipeline. This thesis work has explored these methods of utilizing convolutional neural networks to extract information from images and builds upon it by exploring a set of machine learning algorithms that replace the more commonly used SoftMax function as a classifier, in attempts to further increase not only the accuracy but also optimize the use of computational resources. The work also explores different techniques for the face detection subtask in the pipeline by comparing two approaches. One of these approaches is more frequently used in the state-of-the-art and is said to be more viable for possible real-time applications, namely the Viola-Jones algorithm. The other is a deep learning approach using a state-of-the-art convolutional neural network to perform the detection, in many cases speculated to be too computationally intense to run in real-time. By applying a state-of-the-art inspired new developed convolutional neural network together with the SoftMax classifier, the final performance did not reach state-of-the-art accuracy. However, the machine-learning classifiers used shows promise and bypass the SoftMax function in performance in several cases when given a massively smaller number of samples as training. Furthermore, the results given from implementing and testing a pure deep learning approach, using deep learning algorithms for both the detection and classification stages of the pipeline, shows that deep learning might outperform the classic Viola-Jones algorithm in terms of both detection rate and frames per second.

La détection de peau consiste à détecter les pixels correspondant à une peau humaine dans une image couleur. Les visages constituent une catégorie de stimulus importante par la richesse des informations qu’ils véhiculent car avant de reconnaître n’importe quelle personne il est indispensable de localiser et reconnaître son visage. La plupart des applications liées à la sécurité et à la biométrie reposent sur la détection de régions de peau telles que la détection de visages, le filtrage d'objets 3D pour adultes et la reconnaissance de gestes. En outre, la détection de la saillance des mailles 3D est une phase de prétraitement importante pour de nombreuses applications de vision par ordinateur. La segmentation d'objets 3D basée sur des régions saillantes a été largement utilisée dans de nombreuses applications de vision par ordinateur telles que la correspondance de formes 3D, les alignements d'objets, le lissage de nuages de points 3D, la recherche des images sur le web, l’indexation des images par le contenu, la segmentation de la vidéo et la détection et la reconnaissance de visages. La détection de peau est une tâche très difficile pour différentes raisons liées en général à la variabilité de la forme et la couleur à détecter (teintes différentes d’une personne à une autre, orientation et tailles quelconques, conditions d’éclairage) et surtout pour les images issues du web capturées sous différentes conditions de lumière. Il existe plusieurs approches connues pour la détection de peau : les approches basées sur la géométrie et l’extraction de traits caractéristiques, les approches basées sur le mouvement (la soustraction de l’arrière-plan (SAP), différence entre deux images consécutives, calcul du flot optique) et les approches basées sur la couleur. Dans cette thèse, nous proposons des méthodes d'optimisation numérique pour la détection de régions de couleurs de peaux et de régions saillantes sur des maillages 3D et des nuages de points 3D en utilisant un graphe pondéré. En se basant sur ces méthodes, nous proposons des approches de détection de visage 3D à l'aide de la programmation linéaire et de fouille de données (Data Mining). En outre, nous avons adapté nos méthodes proposées pour résoudre le problème de la simplification des nuages de points 3D et de la correspondance des objets 3D. En plus, nous montrons la robustesse et l’efficacité de nos méthodes proposées à travers de différents résultats expérimentaux réalisés. Enfin, nous montrons la stabilité et la robustesse de nos méthodes par rapport au bruit. / Skin detection involves detecting pixels corresponding to human skin in a color image. The faces constitute a category of stimulus important by the wealth of information that they convey because before recognizing any person it is essential to locate and recognize his face. Most security and biometrics applications rely on the detection of skin regions such as face detection, 3D adult object filtering, and gesture recognition. In addition, saliency detection of 3D mesh is an important pretreatment phase for many computer vision applications. 3D segmentation based on salient regions has been widely used in many computer vision applications such as 3D shape matching, object alignments, 3D point-point smoothing, searching images on the web, image indexing by content, video segmentation and face detection and recognition. The detection of skin is a very difficult task for various reasons generally related to the variability of the shape and the color to be detected (different hues from one person to another, orientation and different sizes, lighting conditions) and especially for images from the web captured under different light conditions. There are several known approaches to skin detection: approaches based on geometry and feature extraction, motion-based approaches (background subtraction (SAP), difference between two consecutive images, optical flow calculation) and color-based approaches. In this thesis, we propose numerical optimization methods for the detection of skins color and salient regions on 3D meshes and 3D point clouds using a weighted graph. Based on these methods, we provide 3D face detection approaches using Linear Programming and Data Mining. In addition, we adapted our proposed methods to solve the problem of simplifying 3D point clouds and matching 3D objects. In addition, we show the robustness and efficiency of our proposed methods through different experimental results. Finally, we show the stability and robustness of our methods with respect to noise.

Made available in DSpace on 2015-05-14T12:36:43Z (GMT). No. of bitstreams: 1 arquivototal.pdf: 2151124 bytes, checksum: ffc486a2022781c4365766e4bf1e7054 (MD5) Previous issue date: 2009-08-27 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Human face has a very complex and variable pattern, which makes the face detection and recognition operations a challenging problem. The scope of these operations is quite comprehensive, involving mainly security applications, such as authorization for physical and logical access, people tracking, and real time authentication. In addition to security applications, face detection and recognition can also be associated with other applications, such as human-computer interaction and virtual reality. Several studies of face detection and recognition have been proposed and developed by researchers, pursuing greater precision and efficiency. Currently there are face detectors and recognizers with accuracy exceeding 95%. Commercial systems are available as well. This work presents a study on several face detection and recognition methods. Also was discussed the possibility of developing a new face detection method using Prediction by Partial Match (PPM), Entropy and Discrete Cosine Transform (DCT). It is further proposed a new face recognition method based on DCT. Finally, is proposed an architecture for a face detection and recognition system in video. To validate the architecture, the proposed system was implemented using one of the best detectors in the literature and the recognizer produced in this work. Several experiments were performed, and both the face detector used as the recognizer developed were effective, achieving success rates compatible with most current methods / A face humana possui um padrão bastante complexo e variável, o que torna as operações de detecção e reconhecimento de face um problema desafiador. O campo de aplicação dessas operações é bastante abrangente, envolvendo principalmente aplicações de segurança, como autorização de acesso físico e lógico, rastreamento de pessoas e autenticação em tempo real. Além de aplicações de segurança, a detecção e o reconhecimento de faces também pode ser associado a outras aplicações, como interação homem-máquina e realidade virtual. Diversos trabalhos de detecção e reconhecimento de face vêm sendo propostos e desenvolvidos pela comunidade científica, buscando continuamente uma maior precisão e eficiência. Atualmente já estão disponíveis detectores e reconhecedores de face com precisão superior a 95%. Sistemas comerciais também já estão disponíveis no mercado. Este trabalho apresenta um estudo sobre os diversos métodos de detecção e reconhecimento de face existentes. Também foi analisada a possibilidade de desenvolvimento de um novo método de detecção de face utilizando Predição por Casamento Parcial (Prediction by Partial Match, PPM), Entropia e Transformada Cosseno Discreta (Discrete Cosine Transform, DCT). Propõe-se ainda, um novo método de reconhecimento de face baseado na DCT. Por fim, apresenta-se a arquitetura de um sistema de detecção e reconhecimento de face em vídeo. Para validação desta arquitetura, o sistema proposto foi implementado utilizando um dos melhores detectores encontrados na literatura e o reconhecedor produzido neste trabalho. Diversos experimentos foram realizados e tanto o detector de face utilizado, quanto o reconhecedor desenvolvido mostraram-se eficientes, atingindo taxas de acerto compatíveis com os métodos mais atuais.

O crescimento da utilização de sistemas de reconhecimento no mundo contemporâneo exige processos de detecção cada vez mais robustos e ágeis. Aplicáveis desde sistemas de teleconferência empresarial até mecanismos de segurança e vigilância, a detecção e o reconhecimento de pessoas tornaram-se uma constante. Na tentativa de buscar caminhos alternativos, tanto para os problemas de detecção, quanto para os de reconhecimento, este trabalho propõe a utilização de medidas biométricas, mensuradas em imagens digitalizadas de faces humanas. A partir do estudo de tais medidas, torna-se possível a verificação de proporções existentes na face, especialmente a proporção divina, podendo constituir, no futuro, a base para algoritmos de detecção e/ou reconhecimento que usufruam das informações trazidas por tais proporções. Diante de uma reduzida quantidade de publicações no meio científico que utilizam a proporção divina como meio de detecção e/ou reconhecimento em processamento de imagens, esta investigação vem contribuir com alguns passos nessa direção / The increase of recognition systems in the contemporary world has demanded robust and agile detection processes. From teleconference systems to security and monitoring mechanisms, the detection and recognition of people have became constantly used and applied. In attempt to search for alternative ways to solve both detection and recognition problems, this work proposes the utilization of biometric measures, taken in digital image of human faces. From the study of such measures, it’s possible to verify face proportions, especially the divine proportion, which could allows, in the future, to implement the detection and/or recognition algorithms that utilize such proportions. Due to small amount of scientific publications that use the divine proportion as a way of detection and/or recognition in image processing, this investigation contributes with some steps in this direction

This thesis attempts to create an application to determine head rotation angle in video recorded from a security camera. The application consists of three parts: face detection, facial landmarks detection and determination of person's head rotation. The face detection has been implemented using Viola-Jones and HOG algorithms. Facial landmarks detection has been done using algorithm based on active shape model. Two methods to calculate the head rotation angles have been used: the first method works with anthropometric head features. The second method uses Perspective-n-Point algorithm to find the right rotation angles. Finally, all algorithms implemented have been tested and the proper parameters have been determined.

Táto diplomová práca sa venuje automatickému rozpoznávaniu ludí v obrazových dátach s využitím konvolučných neurónových sieti na určenie polohy tváre a následnej analýze získaných dát. Výsledkom analýzy tváre je určenie pohlavia, emócie a veku osoby. Práca obsahuje popis použitých architektúr konvolučných sietí pre každú podúlohu. Sieť na odhad veku má natrénované nové váhy, ktoré sú vzápätí zmrazené a majú do svojej architektúry vložené LSTM vrstvy. Tieto vrstvy sú samostatne dotrénované a testované na novom datasete vytvorenom pre tento účel. Výsledky testov ukazujú zlepšenie predikcie veku. Riešenie pre rýchlu, robustnú a modulárnu detekciu tváre a ďalších ludských rysov z jedného obrazu alebo videa je prezentované ako kombinácia prepojených konvolučných sietí. Tieto sú implementované v podobe skriptu a následne vysvetlené. Ich rýchlosť je dostatočná pre ďalšie dodatočné analýzy tváre na živých obrazových dátach.