Aim: To evaluate the ability of a publicly available facial recognition application program interface (API) to calculate similarity scores for pre- and post-surgical photographs of patients undergoing orthognathic surgeries. Our primary objective was to identify which surgical procedure(s) had the greatest effect(s) on similarity score. Methods: Standard treatment progress photographs for 25 retrospectively identified, orthodontic-orthognathic patients were analyzed using the API to calculate similarity scores between the pre- and post-surgical photographs. Photographs from two pre-surgical timepoints were compared as controls. Both relaxed and smiling photographs were included in the study to assess for the added impact of facial pose on similarity score. Surgical procedure(s) performed on each patient, gender, age at time of surgery, and ethnicity were recorded for statistical analysis. Nonparametric Kruskal-Wallis Rank Sum Tests were performed to univariately analyze the relationship between each categorical patient characteristic and each recognition score. Multiple comparison Wilcoxon Rank Sum Tests were performed on the subsequent statistically significant characteristics. P-Values were adjusted for using the Bonferroni correction technique. Results: Patients that had surgery on both jaws had a lower median similarity score, when comparing relaxed expressions before and after surgery, compared to those that had surgery only on the mandible (p = 0.014). It was also found that patients receiving LeFort and bilateral sagittal split osteotomies (BSSO) surgeries had a lower median similarity score compared to those that received only BSSO (p = 0.009). For the score comparing relaxed expressions before surgery versus smiling expressions after surgery, patients receiving two-jaw surgeries had lower scores than those that had surgery on only the mandible (p = 0.028). Patients that received LeFort and BSSO surgeries were also found to have lower similarity scores compared to patients that received only BSSO when comparing pre-surgical relaxed photographs to post-surgical smiling photographs (p = 0.036). Conclusions: Two-jaw surgeries were associated with a statistically significant decrease in similarity score when compared to one-jaw procedures. Pose was also found to be a factor influencing similarity scores, especially when comparing pre-surgical relaxed photographs to post-surgical smiling photographs.

Introdução: O espasmo hemifacial (EHF) é um distúrbio do movimento caracterizado por contrações tônicas e/ou clônicas, involuntárias e irregulares, dos músculos de uma hemiface inervados pelo nervo facial ipsilateral. Por acometer a face, região em que majoritariamente expressamos nossas emoções, uma doença que altere a sua motricidade fisiológica pode gerar dificuldades não só de expressão mas também de interpretação das emoções faciais, com consequente prejuízo psicossocial a estes indivíduos. Objetivo: avaliar a capacidade de reconhecimentos facial de emoções dos indivíduos com EHF do ambulatório de toxina botulínica do Hospital de Clínicas da USP de Ribeirão Preto. Métodos: 51 pacientes com EHF foram selecionados e avaliados quanto à sua capacidade de reconhecimento facial das 6 emoções básicas propostas por Ekman, através de um teste dinâmico preto e branco exposto em um programa de computador em uma tela touch screen. Os resultados foram analisados e comparados com outros 51 indivíduos do grupo controle, pareados por idade, sexo e escolaridade. Resultados: De maneira geral, verificou-se que indivíduos com pior frequência e gravidade dos espasmos hemifaciais avaliados pela escala de Jankovic apresentaram maior número de acertos sobre a emoção \"nojo\" e aqueles com pior assimetria facial pela escala de Sunnybrook, tiveram menor número de respostas corretas para a emoção \"surpresa\". Conclusão: Pacientes com EHF mais graves tendem a denotar mais facilmente a emoção \"nojo\" e aqueles com faces mais assimétricas, tem maior dificuldade em denotaram a emoção \"surpresa\". Isso pode corroborar para possíveis prejuízos psicossociais na vida destes pacientes, devendo o neurologista assistente atentar-se não somente para o distúrbio do movimento que se apresenta, mas também por seus possíveis efeitos deletérios no convívio social daqueles / Introduction: Hemifacial spasm (HFS) is a movement disorder characterized by involuntary and irregular tonic and / or clonic contractions of the muscles of a hemiface innervated by the ipsilateral facial nerve. Because it affects the face, the region in which we express our emotions, a disease that affects physiological motility may lead to difficulties not only in expression but also in the interpretation of facial emotions, with consequent psychosocial damage to these individuals. Objective: to evaluate the facial recognition abilities of individuals with FHD at Botulinum toxin outpatient clinic of Clinical Hospital, USP, Ribeirão Preto. Methods: Fifty-one patients with HFS were selected and evaluated for their facial recognition ability of the six basic emotions proposed by Ekman through a dynamic black-and-white test exposed in a computer on a touch screen. The results were analyzed and compared with 51 other individuals in the control group, matched by age, sex and schooling. Results: In general, it was verified that individuals with worse frequency and severity of the hemifacial spasms evaluated by the Jankovic scale had a greater number of correct answers about the emotion \"disgusted\" and those with worse facial asymmetry by the Sunnybrook scale, had less number of correct answers to the \"surprise\" emotion. Conclusion: Patients with EHF of worse severity and frequency tend to perceive more easily the \"disgust\" emotion and those with more asymmetrical faces, have more difficulty in denoting the \"surprise\" emotion. This may corroborate possible psychosocial impairments in the life of these patients, and the assistant neurologist should be aware not only of the movement disorder that is present, but also of their possible deleterious effects on their social life

Automatic face recognition has attracted the attention of many research institutes, commercial industries, and government agencies in the past few years mainly due to the emergence of numerous applications, such as surveillance, access control to secure facilities, and airport screening. Almost all of the research on the early days of face recognition was focused on using 2-D (intensity/portrait) images of the face. While several sophisticated 2-D solutions have been proposed, unbiased evaluation studies show that their collective performance remains unsatisfactory, and degrades significantly with variations in lighting condition, face position, makeup, or existence of non-neutral facial expressions. Recent developments in 3-D imaging technology has made cheaper, quicker and more reliable acquisition of 3-D facial models a reality. These 3-D facial models contain information about the anatomical structure of the face that remains constant under variable lighting conditions, facial makeup, and pose variations. Thus, researchers are considering to utilize 3-D structure of the face alone or in combination with 2-D information to alleviate inherent limitations of 2-D images and attain better performance. Published 3-D face recognition algorithms have demonstrated promising results confirming the effectiveness of 3-D facial models in dealing with the above mentioned factors contributing to the failure of 2-D face recognition systems. However, the majority of these 3-D algorithms are extensions of conventional 2-D approaches, where intensity images are simply replaced by 3-D models rendered as range images. These algorithms are not specifically tailored to exploit abundant geometric and anthropometric clues available in 3-D facial models. In this dissertation we introduce innovative 3-D and 2-D+3-D facial measurements (features) that effectively describe the geometric characteristics of the corresponding faces. Some of the features described in this dissertation, as well as many features proposed in the literature are defined around or between meaningful facial landmarks (fiducial points). In order to reach our goal of designing an accurate automatic face recognition system, we also propose a novel algorithm combining 3-D (range) and 2-D (portrait) Gabor clues to pinpoint a number of points with meaningful anthropometric definitions with significantly better accuracies than those achievable using a single modality alone. This dissertation is organized as follows. In Chapter 1, various biometric modalities are introduced and the advantages of the facial biometrics over other modalities are discussed. The discussion in Chapter 1 is continued with introduction of the face recognition’s modes of operation followed by some current and potential future applications. The problem statement of this dissertation is also included in this chapter. In Chapter 2, an extensive review of the successful 2-D, 3-D, and 2-D+3-D face recognition algorithms are provided. Chapter 3 presents the details of our innovative 3-D and 2-D+3-D face features, as well as our accurate fiducial point detection algorithm. Conclusions and directions for future extensions are presented in Chapter 4. / text

Cette thèse s'intéresse aux biométries dites douces, et notamment à leurs utilisations en termes de sécurité, dans le cadre de différents scénarii commerciaux, incluant des aspects usage. L'accent sera ainsi porté sur les caractéristiques faciales qui constituent un jeu de traits significatifs de l'apparence physique mais aussi comportementale de l'utilisateur permettant de différencier, classer et identifier les individus. Ces traits, qui sont l'âge, le sexe, les cheveux, la peau et la couleur des yeux, mais aussi la présence de lunettes, de moustache ou de barbe, comportent plusieurs avantages notamment la facilité avec laquelle ils peuvent être acquis, mais également du fait qu'ils correspondent à la façon dont les êtres humains perçoivent leurs environnements. Plus précisément, les traits issus de la biométrie douce sont compatibles avec la manière dont l'humain tend à catégoriser son entourage, une démarche impliquant une structuration hiérarchique des différents traits. Cette thèse explore ces différents traits et leurs applications dans les systèmes de biométries douces (SBS), et met l'accent sur la manière dont de tels systèmes peuvent atteindre des buts différents, y compris la recherche accélérée dans des bases de données, l'identification et la ré-identification d'individus, mais également la prédiction et la quantification de l'esthétique d'un visage. Ce travail est motivé notamment par l'importance croissante de ces applications dans notre société en constante évolution, mais aussi par le côté peu contraignant du système. En effet, les SBS sont généralement non-intrusifs, et nécessitent le plus souvent de faibles temps de calculs, permettant ainsi une analyse biométrique rapide, sans imposer obligatoirement l'accord et la coopération de l'individu. Ces atouts rendent la biométrie douce indispensable dans les applications qui ont besoin de traitement d'images ou de vidéos en temps réel.

Adults are experts at assessing emotions, an ability essential for appropriate social interaction. The present study, investigated this ability’s development, examining infants’ matching of facial and body emotional information. In Experiment 1, 18 6.5-month-olds were familiarized to angry or happy bodies or faces. Those familiarized to bodies were tested with familiar and novel emotional faces. Those habituated to faces were tested with bodies. The 6.5-month-old infants exhibited a preference for the familiar emotion, matching between faces and bodies. In Experiment 2, 18 6.5-month-olds were tested with faces and bodies displaying anger and sadness. Infants familiarized to faces showed a familiarity preference; Infants familiarized to bodies failed to discriminate. Thus, infants generalized from faces to bodies, but failed in the reverse. A follow-up study increased the duration of familiarization: 12 additional 6.5-month-olds were exposed to two-30s familiarizations with bodies, and tested with faces. Additional exposure induced matching of emotions. In Experiment 3, 18 3.5-month-olds were tested using Experiment 1’s stimuli and methodology. The 3.5-month-old infants did not discriminate during test trials. These results suggest 6.5-month-old infants are capable of matching angry, sad and happy faces and bodies. However, 3.5-month-olds are not, suggesting a developmental change between 3.5- and 6.5-months.

There has been considerable amount of work done in the past decade by different biometric vendors as well as researchers in order to improve the accuracy of both biometric data collection systems as well as the underlying algorithms to perform different mapping and matching. Biometric Facial recognition systems are continuously improving with advancements in technology as well as decreasing hardware costs. However, in order to develop efficient biometric facial recognition systems, organizations must take into account different technical issues which include reliability, usability, interoperability as well as data collection, dissemination and evaluation of such systems. Adherence to standards is a key for interoperable systems and user acceptance along with trust can lead to easy deployment of such facial biometric systems in a society. These technical as well as social issues were identified by regular discussions with the Chairman and members of Swedish National Biometric Association’s and conducting interview sessions with different biometric companies who are specially dealing with face recognition biometric systems and are supplemented with a thorough literature review of the state of art research practices within face recognition domain. / The demand for user-friendly, reliable, proactive, high performance and highly interoperable biometric devices has grown exponentially. After conducting the case study, collecting data through the questionnaire and reviewing the literature the researchers come to the point that organizations willing to develop or deploy such systems needs to take into account many technical as well as social issues to make the best out of these systems. The extension of face recognition biometric technologies raises new and challenging questions in a society that enshrines privacy and liberty. This system can either enhance or undermine our liberties depending upon its use. A framework is needed by which we may resolve issues pertaining to proper and legitimate deployment and get expected outcome. The framework or centralized structure is needed which will compile and common issues and provide guidelines to face recognition biometric industry.

With facial recognition technology becoming a greater part of our everyday lives the ethicalimplications it may bring is something worth exploring, and according to scientists the technology could already be incorporated in video surveillance. Previous studies have shown thatwomen and men see things differently within this context, something that will be explored inthis paper. Through focus group interviews and a survey study the students could not confirmthis statement, and found that in only one specific context women and men had significantdifferent standpoints. Further the study found that there is a fear of what this technology couldresult in, in the form of abuse of the information gathered about people and how society couldchange with the incorporation of said technology in video surveillance. / Eftersom ansiktsigenkänningsteknologi blir en större del av vår vardag är de etiska konsekvenserna det kan ge något som är värt att utforska, och enligt forskare kan tekniken redanfinnas implementerad i videoövervakning. Tidigare studier har visat att kvinnor och män uppfattar saker annorlunda inom detta sammanhang, något som kommer att undersökas i dennauppsats. Genom fokusgruppsintervjuer och en enkätstudie kunde studenterna inte bekräftadetta uttalande och fann endast att kvinnor och män i ett specifikt sammanhang hade märkbartolika uppfattningar. Vidare fann studien att det finns en rädsla för vad denna teknik kan resultera i, i form av missbruk av den information som samlats in om människor och hur samhälletkan förändras genom införandet av nämnd teknik i videoövervakning.

Vittnesmål spelar en avgörande roll som bevisning vid brottmål och att vittna utgör en viktig del i att kunna klara upp brott. Att ögonvittnen identifierar en person är en vanlig metod för att få bevismaterial i en rättegång eller för att vägleda polisen i utrednings- och spaningssammanhang. Ögonvittnesidentifiering av ansikten är dock inte en helt säker metod då det mänskliga minnet besitter begränsningar och är sårbart för suggestioner vilket gör det nödvändigt att undersöka samband mellan kognitiva faktorer och förmågan att korrekt identifiera ansikten. Detta i syfte att minska antalet felaktiga identifikationer av oskyldiga. Syftet med studien var att besvara frågan om förmågan till ansiktsigenkänning kan prediceras av kognitiva förmågor. Studien har nyttjat insamlad data från den longitudinella studien Betula(Nilsson m.fl., 1997) som startade 1988 i Umeå. Med ett urval om 1 715 personer i åldrarna35–85 år (M = 58.22, SD = 13.79) har resultatet från nio olika kognitiva tester utforskats i förhållande till ett ansiktsigenkänningstest bestående av barnansikten. Genom korrelationsanalyser och stegvisa regressionsanalyser framkom att förmågan till ansiktsigenkänningsförmåga kan prediceras till viss del. Den starkaste prediktorn för ansiktsigenkänning var visuospatial förmåga och flytande intelligens. Förmågan att känna igen ansikten verkar inte försämras vid högre ålder, vilket gäller för andra kognitiva förmågor. / Testimony plays a crucial role in the work of presenting evidence in criminal cases and to solve crimes. Eyewitness to identification is commonly used method to collect evidence for a trial or to aid the police in their investigations. However, identification of faces is not a completely reliable method because the human memory has limitations and is vulnerable to suggestions, which stresses the importance of investigating relationships between cognitive factors and the ability to correctly identify faces. This in order to reduce the number of incorrect identifications of innocent people. Thus, the purpose of this study was to answer the question “whether theability to recognize faces can be predicted by cognitive abilities”. This study has made use of data collected within the longitudinal Betula study (Nilsson et al, 1997) which started in Umeå in 1988. With a sample of 1 715 participants in aged 35-85 years (M = 58.22, SD = 13.79)results from nine different cognitive tests were explored in relation to performance in facerecognition task consisting of children’s faces. Through correlation analyzes and stepwiseregression analyzes, it emerged that the ability to face recognition ability can be predicted tosome extent. The strongest predictor of facial recognition was visuospatial ability and fluid intelligence. The ability to recognize faces does not seem to deteriorate at an older age, required for other cognitive abilities.

Forged videos of swapped faces, so-called deepfakes, have gained a  lot  of  attention in recent years. Methods for automated detection of this type of manipulation are also seeing rapid progress in their development. The purpose of this thesis work is to evaluate the possibility and effectiveness of using deep embeddings from facial recognition networks as base for detection of such deepfakes. In addition, the thesis aims to answer whether or not the identity embeddings contain information that can be used for detection while analyzed over time and if it is suitable to include information about the person's head pose in this analysis. To answer these questions, three classifiers are created with the intent to answer one question each. Their performances are compared with each other and it is shown that identity embeddings are suitable as a basis for deepfake detection. Temporal analysis of the embeddings also seem effective, at least for deepfake methods that only work on a frame-by-frame basis. Including information about head poses in the videos is shown to not improve a classifier like this.

Emotions are thought to play a crucial role in food behavior. Non-rational emotional decision making may be credited as the reason why consumers select what, how, and when they choose to interact with a food product. In this research, three experiments were completed for the overall goal of understanding the usefulness and validity of selected emotional measurement tools, specifically emotion questionnaire ballots and facial expression analysis, as compared to conventional sensory methods in developing a holistic view of product interest and engagement. Emotional response to 1% low-fat unflavored and chocolate-flavored milk was evaluated by using an emotion-based questionnaire as well as facial expression analysis software, to measure post-experience cognitive and in-the-moment intrinsic (implicit) emotional response, respectively. The software correlated facial movements of participants to associated basic emotions to estimate with what degree consumers were expressing these measured emotions upon presentation of each sample. Finally, the adapted facial expression method was compared to expected measurements from previous studies by measuring emotional facial response to four (sweet, salt, sour, and bitter) basic tastes. The cognitive emotion ballot and implicit facial analysis were able to differentiate between milk samples and offer a greater understanding of the consumer experience. Validity of the facial expression method was lacking for reasons including high individual taste variability, social context, intensities of stimuli, quality of video data capture, calibration settings, sample size number, analysis duration, and software sensitivity limitations. To better validate automatic facial expression methodology, further study is needed to investigate and minimize method limitations. / Master of Science in Life Sciences