Faces are among the most complex stimuli that the human visual system processes. Growing commercial interest in face recognition is encouraging, but it also turns out to be a challenging endeavour. These challenges arise when the situations are complex and cause varied facial appearance due to e.g., occlusion, low-resolution, and ageing. The problem of computer-based face recognition using partial facial data is still largely an unexplored area of research and how does computer interpret various parts of the face. Another challenge is age progression and regression, which is considered to be the most revealing topic for understanding the human face changes during life. In this research, the various computational face recognition models are investigated to overcome the challenges posed by ageing and occlusions/partial faces. For partial face-based face recognition, a pre-trained VGGF model is employed for feature extraction and then followed by popular classifiers such as SVMs and Cosine Similarity CS for classification. In this framework, parts of faces such as eyes, nose, forehead, are used individually for training and testing. The results showing that there is an improvement in recognition in small parts, such as recognition rate in forehead enhanced form about 0% to nearly 35%, eyes from about 22% to approximately 65%. In the second framework, five sub-models were built based on Convolutional Neural Networks (CNNs) and those models are named Eyes-CNNs, Nose-CNNs, Mouth-CNNs, Forehead-CNNs, and combined EyesNose-CNNs. The experimental results illustrate a high recognition rate when it comes to small parts, for example, eyes increased up to about 90.83% and forehead reached about 44.5%. Furthermore, the challenge of face ageing is also approached by proposing an age-template based framework, generating an age-based face template for enhanced face generation and recognition. The results showing that generated new aged faces are more reliable comparing with state-of-the-art.

Yes / Face recognition is a very current subject of great interest in the area of visual computing. In the past, numerous face recognition and authentication approaches have been proposed, though the great majority of them use full frontal faces both for training machine learning algorithms and for measuring the recognition rates. In this paper, we discuss some novel experiments to test the performance of machine learning, especially the performance of deep learning, using partial faces as training and recognition cues. Thus, this study sharply differs from the common approaches of using the full face for recognition tasks. In particular, we study the rate of recognition subject to the various parts of the face such as the eyes, mouth, nose and the forehead. In this study, we use a convolutional neural network based architecture along with the pre-trained VGG-Face model to extract features for training. We then use two classifiers namely the cosine similarity and the linear support vector machine to test the recognition rates. We ran our experiments on the Brazilian FEI dataset consisting of 200 subjects. Our results show that the cheek of the face has the lowest recognition rate with 15% while the (top, bottom and right) half and the 3/4 of the face have near 100% recognition rates. / Supported in part by the European Union's Horizon 2020 Programme H2020-MSCA-RISE-2017, under the project PDE-GIR with grant number 778035.

Diese kumulative Dissertation umfasst zwei Veröffentlichungen zu drei Bereichen der Emotionskommunikation. Ziel war es, zu untersuchen, ob OP-Masken die Emotionswahrnehmung, die Affiliation und die emotionale Mimikry (d.h. die automatische, aber zielabhängige Imitation des emotionalen Ausdrucks von Interaktionspartner:innen) reduzieren. In zwei Online-Experimenten (Studie 1: N=200, britische Stichprobe; Studie 2: N=235, deutsche Stichprobe) wurden subjektive Bewertungen und die emotionale Mimikry als Reaktion auf maskierte und unmaskierte Gesichter untersucht. Die wahrgenommene Emotionsintensität und die Genauigkeit der Emotionserkennung dienten als Indikatoren für die Emotionswahrnehmung. Die wahrgenommene zwischenmenschliche Nähe diente als Indikator für die Affiliation. Die emotionale Mimikry wurde mit Hilfe einer Gesichtsaktivitätserkennungstechnologie gemessen. In der ersten Studie sahen erwachsene Proband:innen erwachsene Zielpersonen, die Freude und Trauer ausdrückten, eingebettet in Innen- und Außenszenen. In der zweiten Studie sahen erwachsene Proband:innen Erwachsene und Kinder, die Freude, Trauer oder Ärger ausdrückten. Die Freudemimikry wurde durch Masken reduziert, insbesondere wenn die Zielpersonen Kinder waren. Im Gegensatz dazu war die Trauermimikry bei Kindergesichtern stärker und wurde, wie auch die Ärgermimikry, durch Masken nicht beeinträchtigt. Wir konnten auch zeigen, dass durch Gesichtsmasken verringerte Freudewahrnehmung und Nähe mit einer verringerten Freudemimikry verbunden waren. Die Studien zeigen somit, wie erwartet wurde, eine maskenbedingte Verringerung der Emotionswahrnehmung, der Affiliation, und der emotionalen Mimikry. Insgesamt trägt die vorliegende Arbeit zu unserem Verständnis der sozio-affektiven Auswirkungen der partiellen Gesichtsverdeckung bei und stützt die Theorie der emotionalen Mimikry im sozialen Kontext von Hess und Fischer, nach der Emotionswahrnehmung und Affiliation die emotionale Mimikry beeinflussen. / This cumulative doctoral dissertation encompasses two publications on three domains of emotion communication. The goal of the dissertation was to assess if surgical face masks reduce emotion perception, affiliation, and emotional mimicry (i.e., automatic but goal-dependent imitation of an interaction partner's emotional display). We conducted two online experiments (Study 1: N=200, U.K. sample; Study 2: N=235, German sample) that assessed subjective ratings and emotional mimicry in response to masked and unmasked faces. Perceived emotion intensity and emotion recognition accuracy served as indicators of emotion perception. Perceived interpersonal closeness (via the Inclusion of Other in the Self Scale) served as an indicator of affiliation. Emotional mimicry was measured using facial activity recognition technology. We also took into account socio-spatial context effects. In the first study, adult perceivers saw adult targets who expressed happiness and sadness embedded into indoor and outdoor scenes. In the second study, adult perceivers saw adult and child targets who expressed happiness, sadness, and anger. We found that happiness mimicry was reduced by masks, particularly when expressers were children. In contrast, sadness mimicry was stronger for children and, like anger mimicry, unaffected by masks. We also found that reduced emotion perception and closeness due to masks were associated with reduced happiness mimicry. The studies support evidence from before and during the COVID-19 pandemic for mask-related reductions of emotion perception and affiliation. Moreover, the studies provide unprecedented evidence on reductions of emotional mimicry in response to masked faces and child targets. Overall, the present work contributes to our understanding of the socio-affective effects of partial face occlusion and supports emotional mimicry in social context theory by Hess and Fischer, according to which emotion perception and affiliation influence emotional mimicry.