Performance of automatic speech recognition (ASR) systems utilizing only acoustic information degrades significantly in noisy environments such as a car cabins. Incorporating audio and visual information together can improve performance in these situations. This work proposes a lip detection and tracking algorithm to serve as a visual front end to an audio-visual automatic speech recognition (AVASR) system. Several color spaces are examined that are effective for segmenting lips from skin pixels. These color components and several features are used to characterize lips and to train cascaded lip detectors. Pre- and post-processing techniques are employed to maximize detector accuracy. The trained lip detector is incorporated into an adaptive mean-shift tracking algorithm for tracking lips in a car cabin environment. The resulting detector achieves 96.8% accuracy, and the tracker is shown to recover and adapt in scenarios where mean-shift alone fails.

Image segmentation is an important step in image processing. A traditional way how to segment an image is a texture-based segmentation that uses texture features to describe image texture. In this work, Local Binary Patterns (LBP) are used for image texture representation. Texture feature is a histogram of occurences of LBP codes in a small image window. The work also aims to comparison of results of various modifications of Local Binary Patterns and their usability in the image segmentation which is done by unsupervised clustering of texture features. The Fuzzy C-Means algorithm is finally used for the clustering in this work.

Face detection and recognition has been very popular topics. Recently, its applications for mobile devices have gained tremendous attention due to the rapid expansion of the market. Although numerous techniques exist for face detection and recognition, only a few solve realistic challenges under the mobile device application environment. In this thesis, we propose an automatic face authentication system including both face detection and recognition components for mobile device applications by using spatially enhanced Local Binary Patterns (LBP) feature extraction. The first contribution is to propose a fast and accurate face detector by using LBP features and its spatially enhanced variant. The simplicity of LBP ensures low computational complexity and spatially enhanced LBP achieves high accuracy. The second contribution is to propose color based spatially enhanced LBP features for face recognition. The proposed features achieve high accuracy by extracting complementary information from color channels and spatial correlations between LBP features.

Face detection and recognition has been very popular topics. Recently, its applications for mobile devices have gained tremendous attention due to the rapid expansion of the market. Although numerous techniques exist for face detection and recognition, only a few solve realistic challenges under the mobile device application environment. In this thesis, we propose an automatic face authentication system including both face detection and recognition components for mobile device applications by using spatially enhanced Local Binary Patterns (LBP) feature extraction. The first contribution is to propose a fast and accurate face detector by using LBP features and its spatially enhanced variant. The simplicity of LBP ensures low computational complexity and spatially enhanced LBP achieves high accuracy. The second contribution is to propose color based spatially enhanced LBP features for face recognition. The proposed features achieve high accuracy by extracting complementary information from color channels and spatial correlations between LBP features.

L’ostéoporose est une maladie osseuse caractérisée par une perte importante de la masse osseuse et des altérations de la microarchitecture du tissu osseux. Aujourd’hui, en routine clinique, le diagnostic de l’ostéoporose est basé principalement sur une mesure de la densité minérale osseuse qui n’est pas suffisante, car elle doit être accompagnée par une analyse de la qualité de la microarchitecture osseuse. Les travaux présentés dans cette thèse concernent la caractérisation des images de radiographies osseuses pour le diagnostic précoce de l’ostéoporose. Pour ce faire, afin de mieux caractériser la texture osseuse sur radiographie, nous avons introduit une nouvelle technique de prétraitement des données pour réduire les redondances et éliminer le bruit issu des capteurs d’acquisition. Pour la caractérisation, nous avons proposé une nouvelle technique d’analyse inspirée des motifs binaires locaux (Local Binary Patterns, LBP). Le nouveau descripteur, appelé 1DLBP (One Dimensional Local Binary Patterns) s’applique de manière unidimensionnelle. Pour tester l’efficacité de notre approche, nous avons réalisé deux études cliniques où le nouveau descripteur LBP1D est comparé à la méthode classique, LBP afin de classifier des patients ostéoporotiques et des sujets sains. Les pourcentages de classification obtenus ont été améliorés de 72% avec la méthode classique LBP à 91% avec le nouveau descripteur 1DLBP. / Osteoporosis is characterized by a significant loss of bone mass and alterations in the microarchitecture of bone tissue. Actually, in clinical routine the diagnosis of osteoporosis is based mainly on measurement of bone mineral density. It turned out that this is not sufficient, it must be accompanied by an analysis of the microarchitecture of the bone to increase the efficiency of diagnosis. This thesis deals with the characterization of images of bone radiographs for the early diagnosis of osteoporosis. To do this, in order to better characterize the texture of bone radiography, we have introduced a new technique for data preprocessing to reduce redundancy and decrease the effect of the noise resulted by the acquisition sensors. For characterization, we propose a new analysis method inspired from the local binary patterns (LBP). The new descriptor called 1DLBP (One Dimensional Local Binary Patterns) applies in one-dimensionally manner. To evaluate the effectiveness of our approach, we conducted two clinical studies where the new descriptor (1DLBP) is compared with the conventional method (LBP) to classify osteoporotic patients and healthy subjects. The classification scores obtained were enhanced by 72% with the conventional LBP descriptor to 91% with 1DLBP descriptor.

>Magister Scientiae - MSc / This research proposes an approach to recognizing facial expressions in the presence of rotations and partial occlusions of the face. The research is in the context of automatic machine translation of South African Sign Language (SASL) to English. The proposed method is able to accurately recognize frontal facial images at an average accuracy of 75%. It also achieves a high recognition accuracy of 70% for faces rotated to 60◦. It was also shown that the method is able to continue to recognize facial expressions even in the presence of full occlusions of the eyes, mouth and left/right sides of the face. The accuracy was as high as 70% for occlusion of some areas. An additional finding was that both the left and the right sides of the face are required for recognition. As an addition, the foundation was laid for a fully automatic facial expression recognition system that can accurately segment frontal or rotated faces in a video sequence.

Visual recognition is the problem of learning visual categories from a limited set of samples and identifying new instances of those categories, the problem is often separated into two types: the specific case and the generic category case. In the specific case the objective is to identify instances of a particular object, place or person. Whereas in the generic category case we seek to recognize different instances that belong to the same conceptual class: cars, pedestrians, road signs and mugs. Specific object recognition works by matching and geometric verification. In contrast, generic object categorization often includes a statistical model of their appearance and/or shape.This thesis proposes a computer vision system for detecting and tracking multiple targets in videos. A preliminary work of this thesis consists on the adaptation of color according to lighting variations and relevance of the color. Then, literature shows a wide variety of tracking methods, which have both advantages and limitations, depending on the object to track and the context. Here, a deterministic method is developed to automatically adapt the tracking method to the context through the cooperation of two complementary techniques. A first proposition combines covariance matching for modeling characteristics texture-color information with optical flow (KLT) of a set of points uniformly distributed on the object . A second technique associates covariance and Mean-Shift. In both cases, the cooperation allows a good robustness of the tracking whatever the nature of the target, while reducing the global execution times .The second contribution is the definition of descriptors both discriminative and compact to be included in the target representation. To improve the ability of visual recognition of descriptors two approaches are proposed. The first is an adaptation operators (LBP to Local Binary Patterns ) for inclusion in the covariance matrices . This method is called ELBCM for Enhanced Local Binary Covariance Matrices . The second approach is based on the analysis of different spaces and color invariants to obtain a descriptor which is discriminating and robust to illumination changes.The third contribution addresses the problem of multi-target tracking, the difficulties of which are the matching ambiguities, the occlusions, the merging and division of trajectories.Finally to speed algorithms and provide a usable quick solution in embedded applications this thesis proposes a series of optimizations to accelerate the matching using covariance matrices. Data layout transformations, vectorizing the calculations (using SIMD instructions) and some loop transformations had made possible the real-time execution of the algorithm not only on Intel classic but also on embedded platforms (ARM Cortex A9 and Intel U9300).

Abstract Local binary patterns (LBP) are among the most popular image description methods and have been successfully applied in a diverse set of computer vision problems, covering texture classification, material categorization, face recognition, and image segmentation, to name only a few. The popularity of the LBP methodology can be verified by inspecting the number of existing studies about its different variations and extensions. The number of those studies is vast. Currently, the methodology has been acknowledged as one of the milestones in face recognition research. The starting point of this research is to gain more understanding of which principles the original LBP descriptor is based on. After gaining some degree of insight, yet another try is made to improve some steps of the LBP pipeline, consisted of image pre-processing, pattern sampling, pattern encoding, binning, and further histogram post-processing. The main contribution of this thesis is a bunch of novel LBP extensions that partly try to unify some of the existing derivatives and extensions. The basis for the design of the new additional LBP methodology is to maximise data-driven premises, at the same time minimizing the need for tuning by hand. Prior to local binary pattern extraction, the thesis presents an image upsampling step dubbed as image pre-interpolation. As a natural consequence of upsampling, a greater number of patterns can be extracted and binned to a histogram improving the representational performance of the final descriptor. To improve the following two steps of the LBP pipeline, namely pattern sampling and encoding, three different learning-based methods are introduced. Finally, a unifying model is presented for the last step of the LBP pipeline, namely for local binary pattern histogram post-processing. As a special case of this, a novel histogram smoothing scheme is proposed, which shares the motivation and the effects with the image pre-interpolation for the most of its part. Deriving descriptors for such face recognition problems as face verification or age estimation has been and continues to be among the most popular domains where LBP has ever been applied. This study is not an exception in that regard as the main investigations and conclusions here are made on the basis of how the proposed LBP variations perform especially in the problems of face recognition. The experimental part of the study demonstrates that the proposed methods, experimentally validated using publicly available texture and face datasets, yield results comparable to the best performing LBP variants found in the literature, reported with the corresponding benchmarks. / Tiivistelmä Paikalliset binäärikuviot kuuluvat suosituimpiin menetelmiin kuville suoritettavassa piirteenirrotuksessa. Menetelmää on sovellettu moniin konenäön ongelmiin, kuten tekstuurien luokittelu, materiaalien luokittelu, kasvojen tunnistus ja kuvien segmentointi. Menetelmän suosiota kuvastaa hyvin siitä kehitettyjen erilaisten johdannaisten suuri lukumäärä ja se, että nykyään kyseinen menetelmien perhe on tunnustettu yhdeksi virstanpylvääksi kasvojentunnistuksen tutkimusalueella. Tämän tutkimuksen lähtökohtana on ymmärtää periaatteita, joihin tehokkaimpien paikallisten binäärikuvioiden suorituskyky perustuu. Tämän jälkeen tavoitteena on kehittää parannuksia menetelmän eri askelille, joita ovat kuvan esikäsittely, binäärikuvioiden näytteistys ja enkoodaus, sekä histogrammin koostaminen ja jälkikäsittely. Esiteltävien uusien menetelmien lähtökohtana on hyödyntää mahdollisimman paljon kohdesovelluksesta saatavaa tietoa automaattisesti. Ensimmäisenä menetelmänä esitellään kuvan ylösnäytteistykseen perustuva paikallisten binäärikuvioiden johdannainen. Ylösnäytteistyksen luonnollisena seurauksena saadaan näytteistettyä enemmän binäärikuvioita, jotka histogrammiin koottuna tekevät piirrevektorista alkuperäistä erottelevamman. Seuraavaksi esitellään kolme oppimiseen perustuvaa menetelmää paikallisten binäärikuvioiden laskemiseksi ja niiden enkoodaukseen. Lopuksi esitellään paikallisten binäärikuvioiden histogrammin jälkikäsittelyn yleistävä malli. Tähän malliin liittyen esitellään histogrammin silottamiseen tarkoitettu operaatio, jonka eräs tärkeimmistä motivaatioista on sama kuin kuvan ylösnäytteistämiseen perustuvalla johdannaisella. Erilaisten piirteenirrotusmenetelmien kehittäminen kasvojentunnistuksen osa-alueille on erittäin suosittu paikallisten binäärikuvioiden sovellusalue. Myös tässä työssä tutkittiin miten kehitetyt johdannaiset suoriutuvat näissä osa-ongelmissa. Tutkimuksen kokeellinen osuus ja siihen liittyvät numeeriset tulokset osoittavat, että esitellyt menetelmät ovat vertailukelpoisia kirjallisuudesta löytyvien parhaimpien paikallisten binäärikuvioiden johdannaisten kanssa.

This thesis addresses the topic of object detection on graphics processing units. As a part of it, a system for object detection using NVIDIA CUDA was designed and implemented, allowing for realtime video object detection and bulk processing. Its contribution is mainly to study the options of NVIDIA CUDA technology and current graphics processing units for object detection acceleration. Also parallel algorithms for object detection are discussed and suggested.

This diploma thesis is focused on object detections through general-purpose computing on graphics processor units. There is an explanation of graphics adapters work and basics of their architecture in this thesis. Based on the adapters, there is the effective work in libraries for general-purpose computing on graphics processor units demonstrated in this thesis. Further, the thesis shows the available algorithms for object detection and which ones from them are possible to be effectively parallelized. In conclusion of this thesis, there is a comparison of the object detections speeds to common implementations on classical processors.