Researchers have been involved for decades in search of an efficient skin detection method. Yet current methods have not overcome the major limitations. To overcome these limitations, in this dissertation, a clustering and region growing based skin detection method is proposed. These methods together with a significant insight result in a more effective algorithm. The insight concerns a capability to define dynamically the number of clusters in a collection of pixels organized as an image. In clustering for most problem domains, the number of clusters is fixed a priori and does not perform effectively over a wide variety of data contents. Therefore, in this dissertation, a skin detection method has been proposed using the above findings and validated. This method assigns the number of clusters based on image properties and ultimately allows freedom from manual thresholding or other manual operations. The dynamic determination of clustering outcomes allows for greater automation of skin detection when dealing with uncertain real-world conditions.

No / As part of the cancer drug development process, evaluation in experimental subcutaneous tumour transplantation models is a key process. This involves implanting tumour material underneath the mouse skin and measuring tumour growth using calipers. This methodology has been proven to have poor reproducibility and accuracy due to observer variation. Furthermore the physical pressure placed on the tumour using calipers is not only distressing for the mouse but could also lead to tumour damage. Non-invasive digital imaging of the tumour would reduce handling stresses and allow volume determination without any potential tumour damage. This is challenging as the tumours sit under the skin and have the same colour pattern as the mouse body making them hard to differentiate in a 2D image. We used the pre-trained convolutional neural network VGG-16 and extracted multiple layers in an attempt to accurately locate the tumour. When using the layer FC7 after RELU activation for extraction, a recognition rate of 89.85% was achieved.

This work addresses the problem of automatically detecting human skin in images without reliance on color information. Unlike previous methods, we present a new approach that performs well in the absence of such information. A key aspect of the work is that color-space augmentation is applied strategically during the training, with the goal of reducing the influence of features that are based entirely on color and increasing more semantic understanding. The resulting system exhibits a dramatic improvement in performance for images in which color details are diminished. We have demonstrated the concept using the U-Net architecture, and experimental results show improvements in evaluations for all Fitzpatrick skin tones in the ECU dataset. We further tested the system with RFW dataset to show that the proposed method is consistent across different ethnicities and reduces bias to any skin tones. Therefore, this work has strong potential to aid in mitigating bias in automated systems that can be applied to many applications including surveillance and biometrics. / Master of Science / Skin segmentation deals with the classification of skin and non-skin pixels and regions in a image containing these information. Although most previous skin-detection methods have used color cues almost exclusively, they are vulnerable to external factors (e.g., poor or unnatural illumination and skin tones). In this work, we present a new approach based on U-Net that performs well in the absence of color information. To be specific, we apply a new color space augmentation into the training stage to improve the performance of skin segmentation system over the illumination and skin tone diverse. The system was trained and tested with both original and color changed ECU dataset. We also test our system with RFW dataset, a larger dataset with four human races with different skin tones. The experimental results show improvements in evaluations for skin tones and complex illuminations.

Face detection, gesture recognition and pornography content assessment are some of the applications that require the detection of human skin in digital imagery. Most methods employ color as the main feature for this task. Whenever the acquisition conditions are controlled, there is available information about illumination, resolution and geometry, making the skin detection problem a relatively easy task for which there are plenty of results in the literature. The problem becomes more challenging in less structured conditions, mainly because of the influence illumination conditions have on the apparent color of objects. There are proposals for color correction that lead to both good and bad classification results, depending on the input data. When dealing with Web imagery, little can be assumed about their content or about the conditions in which they were acquired, and robust techniques are needed for skin detection. This MSc thesis makes a qualitative assessment of seven skin detection models and of four different types of input data. A heuristic is proposed for deciding if an image requires color correction and, if needed, which is the best suited technique. Results are compared by means of measures derived from confusion matrices, and our approach produces competitive classification products. / A detecção de pele humana em imagens digitais é utilizada para diversas aplicações como detecção de faces, reconhecimento de gestos e detecção de pornografia. A forma mais comum de detecção de pele encontrada na literatura é através da cor. A variação de iluminação pode redundar em efeitos nocivos à detecção de pele, pois a aparência da cor de um objeto é diretamente relacionada com a forma em que ele é iluminado. Para a detecção de pele pela cor exclusivamente, estratégias robustas às variações de iluminação e modelos descrevam corretamente o agrupamento das cores da pele devem ser utilizados. Ao enfrentarmos o problema de detecção de pele em ambientes onde não há controle sobre as características da imagem, não encontramos resultados satisfatórios na literatura, principalmente quando se refere à tentativa de minimizar os efeitos da variação de iluminação. As estratégias de correção de cor presentes na literatura melhoram consideravelmente a detecção de pele em algumas situações específicas, mas degradam esta classificação em outras situações. Neste trabalho, avaliamos o desempenho de sete diferentes modelos de detecção de pele, com quatro diferentes tipos de dados de entrada e propusemos uma estratégia para escolha das imagens que serão submetidas à correção de cor e o tipo de técnica de correção de cor mais adequado para esta imagem. A técnica que utiliza um modelo gaussiano bivariado, utilizando as duas primeiras componentes após aplicarmos transformação de componentes principais ao dados RGB da amostra de pele utilizada para treinamento resultou na melhor técnica abordada nesse trabalho ao utilizarmos a correção de cor proposta. Os resultados obtidos são comparados por meio de diversas métricas derivadas da matriz de confusão, e se mostram pelo menos tão bons quanto os alcançados por técnicas disponíveis na literatura.

A detecção de gestos é uma etapa importante em aplicações de interação humanocomputador. Se a mão do usuário é detectada com precisão, tanto a análise quanto o reconhecimento do gesto de mão se tornam mais simples e confiáveis. Neste trabalho, descrevemos um novo método para detecção de pele humana, destinada a ser empregada como uma etapa de pré-processamento para segmentação de gestos de mão em sistemas que visam o seu reconhecimento. Primeiramente, treinamos os modelos de cor e textura de pele (material a ser identificado) a partir de um conjunto de treinamento formado por imagens de pele. Nessa etapa, construímos um modelo de mistura de Gaussianas (GMM), para determinar os tons de cor da pele e um dicionário de textons, para textura de pele. Em seguida, introduzimos um estratégia de fusão estocástica de regiões de texturas, para determinar todos os segmentos de diferentes materiais presentes na imagem (cada um associado a uma textura). Tendo obtido todas as regiões, cada segmento encontrado é classificado com base nos modelos de cor de pele (GMM) e textura de pele (dicionário de textons). Para testar o desempenho do algoritmo desenvolvido realizamos experimentos com o conjunto de imagens SDC, projetado especialmente para esse tipo de avaliação (detecção de pele humana). Comparado com outras técnicas do estado-daarte em segmentação de pele humana disponíveis na literatura, os resultados obtidos em nossos experimentos mostram que a abordagem aqui proposta é resistente às variações de cor e iluminação decorrentes de diferentes tons de pele (etnia do usuário), assim como de mudanças de pose da mão, mantendo sua capacidade de discriminar pele humana de outros materiais altamente texturizados presentes na imagem. / Gesture detection is an important task in human-computer interaction applications. If the hand of the user is precisely detected, both analysis and recognition of hand gesture become more simple and reliable. This work describes a new method for human skin detection, used as a pre-processing stage for hand gesture segmentation in recognition systems. First, we obtain the models of color and texture of human skin (material to be identified) from a training set consisting of skin images. At this stage, we build a Gaussian mixture model (GMM) for identifying skin color tones and a dictionary of textons for skin texture. Then, we introduce a stochastic region merging strategy, to determine all segments of different materials present in the image (each associated with a texture). Once the texture regions are obtained, each segment is classified based on skin color (GMM) and skin texture (dictionary of textons) model. To verify the performance of the developed algorithm, we perform experiments on the SDC database, specially designed for this kind of evaluation (human skin detection). Also, compared with other state-ofthe- art skin segmentation techniques, the results obtained in our experiments show that the proposed approach is robust to color and illumination variations arising from different skin tones (ethnicity of the user) as well as changes of pose, while keeping its ability for discriminating human skin from other highly textured background materials.

A detecção de gestos é uma etapa importante em aplicações de interação humanocomputador. Se a mão do usuário é detectada com precisão, tanto a análise quanto o reconhecimento do gesto de mão se tornam mais simples e confiáveis. Neste trabalho, descrevemos um novo método para detecção de pele humana, destinada a ser empregada como uma etapa de pré-processamento para segmentação de gestos de mão em sistemas que visam o seu reconhecimento. Primeiramente, treinamos os modelos de cor e textura de pele (material a ser identificado) a partir de um conjunto de treinamento formado por imagens de pele. Nessa etapa, construímos um modelo de mistura de Gaussianas (GMM), para determinar os tons de cor da pele e um dicionário de textons, para textura de pele. Em seguida, introduzimos um estratégia de fusão estocástica de regiões de texturas, para determinar todos os segmentos de diferentes materiais presentes na imagem (cada um associado a uma textura). Tendo obtido todas as regiões, cada segmento encontrado é classificado com base nos modelos de cor de pele (GMM) e textura de pele (dicionário de textons). Para testar o desempenho do algoritmo desenvolvido realizamos experimentos com o conjunto de imagens SDC, projetado especialmente para esse tipo de avaliação (detecção de pele humana). Comparado com outras técnicas do estado-daarte em segmentação de pele humana disponíveis na literatura, os resultados obtidos em nossos experimentos mostram que a abordagem aqui proposta é resistente às variações de cor e iluminação decorrentes de diferentes tons de pele (etnia do usuário), assim como de mudanças de pose da mão, mantendo sua capacidade de discriminar pele humana de outros materiais altamente texturizados presentes na imagem. / Gesture detection is an important task in human-computer interaction applications. If the hand of the user is precisely detected, both analysis and recognition of hand gesture become more simple and reliable. This work describes a new method for human skin detection, used as a pre-processing stage for hand gesture segmentation in recognition systems. First, we obtain the models of color and texture of human skin (material to be identified) from a training set consisting of skin images. At this stage, we build a Gaussian mixture model (GMM) for identifying skin color tones and a dictionary of textons for skin texture. Then, we introduce a stochastic region merging strategy, to determine all segments of different materials present in the image (each associated with a texture). Once the texture regions are obtained, each segment is classified based on skin color (GMM) and skin texture (dictionary of textons) model. To verify the performance of the developed algorithm, we perform experiments on the SDC database, specially designed for this kind of evaluation (human skin detection). Also, compared with other state-ofthe- art skin segmentation techniques, the results obtained in our experiments show that the proposed approach is robust to color and illumination variations arising from different skin tones (ethnicity of the user) as well as changes of pose, while keeping its ability for discriminating human skin from other highly textured background materials.

A detecção de gestos é uma etapa importante em aplicações de interação humanocomputador. Se a mão do usuário é detectada com precisão, tanto a análise quanto o reconhecimento do gesto de mão se tornam mais simples e confiáveis. Neste trabalho, descrevemos um novo método para detecção de pele humana, destinada a ser empregada como uma etapa de pré-processamento para segmentação de gestos de mão em sistemas que visam o seu reconhecimento. Primeiramente, treinamos os modelos de cor e textura de pele (material a ser identificado) a partir de um conjunto de treinamento formado por imagens de pele. Nessa etapa, construímos um modelo de mistura de Gaussianas (GMM), para determinar os tons de cor da pele e um dicionário de textons, para textura de pele. Em seguida, introduzimos um estratégia de fusão estocástica de regiões de texturas, para determinar todos os segmentos de diferentes materiais presentes na imagem (cada um associado a uma textura). Tendo obtido todas as regiões, cada segmento encontrado é classificado com base nos modelos de cor de pele (GMM) e textura de pele (dicionário de textons). Para testar o desempenho do algoritmo desenvolvido realizamos experimentos com o conjunto de imagens SDC, projetado especialmente para esse tipo de avaliação (detecção de pele humana). Comparado com outras técnicas do estado-daarte em segmentação de pele humana disponíveis na literatura, os resultados obtidos em nossos experimentos mostram que a abordagem aqui proposta é resistente às variações de cor e iluminação decorrentes de diferentes tons de pele (etnia do usuário), assim como de mudanças de pose da mão, mantendo sua capacidade de discriminar pele humana de outros materiais altamente texturizados presentes na imagem. / Gesture detection is an important task in human-computer interaction applications. If the hand of the user is precisely detected, both analysis and recognition of hand gesture become more simple and reliable. This work describes a new method for human skin detection, used as a pre-processing stage for hand gesture segmentation in recognition systems. First, we obtain the models of color and texture of human skin (material to be identified) from a training set consisting of skin images. At this stage, we build a Gaussian mixture model (GMM) for identifying skin color tones and a dictionary of textons for skin texture. Then, we introduce a stochastic region merging strategy, to determine all segments of different materials present in the image (each associated with a texture). Once the texture regions are obtained, each segment is classified based on skin color (GMM) and skin texture (dictionary of textons) model. To verify the performance of the developed algorithm, we perform experiments on the SDC database, specially designed for this kind of evaluation (human skin detection). Also, compared with other state-ofthe- art skin segmentation techniques, the results obtained in our experiments show that the proposed approach is robust to color and illumination variations arising from different skin tones (ethnicity of the user) as well as changes of pose, while keeping its ability for discriminating human skin from other highly textured background materials.

>Magister Scientiae - MSc / This research proposes an approach to recognizing facial expressions in the presence of rotations and partial occlusions of the face. The research is in the context of automatic machine translation of South African Sign Language (SASL) to English. The proposed method is able to accurately recognize frontal facial images at an average accuracy of 75%. It also achieves a high recognition accuracy of 70% for faces rotated to 60◦. It was also shown that the method is able to continue to recognize facial expressions even in the presence of full occlusions of the eyes, mouth and left/right sides of the face. The accuracy was as high as 70% for occlusion of some areas. An additional finding was that both the left and the right sides of the face are required for recognition. As an addition, the foundation was laid for a fully automatic facial expression recognition system that can accurately segment frontal or rotated faces in a video sequence.

In this paper is tested method for detection skin on the driver's head. It is based on skin color and motion detection. We also tested method for eye detection in image.

The dissertation investigates the promises and challenges for application of cardiac signals in biometrics and affective computing, and noninvasive measurement of cardiac signals. We have mainly discussed two major cardiac signals: electrocardiogram (ECG), and photoplethysmogram (PPG). ECG and PPG signals hold strong potential for biometric authentications and identifications. We have shown that by mapping each cardiac beat from time domain to an angular domain using a limit cycle, intra-class variability can be significantly minimized. This is in contrary to conventional time domain analysis. Our experiments with both ECG and PPG signal shows that the proposed method eliminates the effect of instantaneous heart rate on the shape morphology and improves authentication accuracy. For noninvasive measurement of PPG beats, we have developed a systematic algorithm to extract pulse rate from face video in diverse situations using video magnification. We have extracted signals from skin patches and then used frequency domain correlation to filter out non-cardiac signals. We have developed a novel entropy based method to automatically select skin patches from face. We report beat-to-beat accuracy of remote PPG (rPPG) in comparison to conventional average heart rate. The beat-to-beat accuracy is required for applications related to heart rate variability (HRV) and affective computing. The algorithm has been tested on two datasets, one with static illumination condition and the other with unrestricted ambient illumination condition. Automatic skin detection is an intermediate step for rPPG. Existing methods always depend on color information to detect human skin. We have developed a novel standalone skin detection method to show that it is not necessary to have color cues for skin detection. We have used LBP lacunarity based micro-textures features and a region growing algorithm to find skin pixels in an image. Our experiment shows that the proposed method is applicable universally to any image including near infra-red images. This finding helps to extend the domain of many application including rPPG. To the best of our knowledge, this is first such method that is independent of color cues. / Ph. D. / The heart is an integral part of the human body. With every beat, the heart continuously pumps oxygen-enriched blood to providing fuel to our cells and thus enabling life. The heartbeat is initiated by electrical signals generated in the heart muscles. This electrical activity, which are often governed by our autonomic nervous system, can be measured directly by electrocardiogram (ECG) using advanced and often obtrusive instrumentation. Photoplethysmogram (PPG), on the other hand, measures how the blood volume changes and can be readily measured with inexpensive instrumentation at certain locations (e.g. at the fingertip). The ECG and PPG are widely used cardiac signals in medical science for diagnosis and health monitoring. But, these signals hold greater potential than just its medical diagnostic applications. In this work, we have mainly investigated if these signals can be used to identify an individual. Every human heart differs by their size, shape, locations inside body, and internal structure. This motivated us to represent the signals using a mathematical model and use machine learning algorithm to identify individual persons. We have discussed how our method improves the identification accuracy and can be used with current biometric methods like fingerprint in our phone. The measurement procedures of cardiac signals are often cumbersome and need instruments which may not be available outside medical facilities. Therefore, we have investigated alternative method of remote photoplethysmography (rPPG) that are relatively inexpensive and unobtrusive. In this dissertation, we have used face video of an individual to extract the heart rate information. The flow of blood causes small changes in the color of face skin. This is not visible to human eyes without digital magnification, but we have shown how knowledge of distinct behavior of human heart rate and use of advanced computer vision algorithms helped us to extract vital signals like heart rate with a significant accuracy. In addition, to measure rPPG using face video, we integrated a method for automatic detection of skin from images and videos. Existing skin detection methods depended on color information which is not always available within available video sources. We have developed a novel standalone skin detection method to show that it is not necessary to have color cues for skin detection. Our method relies on the context and the texture based appearance of skin. To the best of our knowledge, this is first such method that is independent of color cues. In summary, the dissertation investigates the promises and challenges for application of cardiac signals in biometrics and nonobtrusive measurement of cardiac signals using face video.