Thesis (M.S.)--State University of New York at Binghamton, Thomas J. Watson School of Engineering and Applied Science, Department of Computer Science, 2008. / Includes bibliographical references.

Indiana University-Purdue University Indianapolis (IUPUI) / This thesis explores a data driven machine learning based solution for Facial reconstruction from three dimensional (3D) skull shape for recognizing or identifying unknown subjects during forensic investigation. With over 8000 unidentified bodies during the past 3 decades, facial reconstruction of disintegrated bodies in helping with identification has been a critical issue for forensic practitioners. Historically, clay modelling has been used for facial reconstruction that not only requires an expert in the field but also demands a substantial amount of time for modelling, even after acquiring the skull model. Such manual reconstruction typically takes from a month to over 3 months of time and effort. The solution presented in this thesis uses 3D Cone Beam Computed Tomography (CBCT) data collected from many people to build a model of the relationship of facial skin to skull bone over a dense set of locations on the face. It then uses this skin-to-bone relationship model learned from the data to reconstruct the predicted face model from a skull shape of an unknown subject. The thesis also extends the algorithm in a way that could help modify the reconstructed face model interactively to account for the effects of age or weight. This uses the predicted face model as a starting point and creates different hypotheses of the facial appearances for different physical attributes. Attributes like age and body mass index (BMI) are used to show the physical facial appearance changes with the help of a tool we constructed. This could improve the identification process. The thesis also presents a methods designed for testing and validating the facial reconstruction algorithm.

Yes / Techniques for facial age progression and regression have many applications and a myriad of challenges. As such, automatic aged or de-aged face generation has become an important subject of study in recent times. Over the past decade or so, researchers have been working on developing face processing mechanisms to tackle the challenge of generating realistic aged faces for applications related to smart systems. In this paper, we propose a novel approach to try and address this problem. We use template faces based on the formulation of an average face of a given ethnicity and for a given age. Thus, given a face image, the target aged image for that face is generated by applying it to the relevant template face image. The resulting image is controlled by two parameters corresponding to the texture and the shape of the face. To validate our approach, we compute the similarity between aged images and the corresponding ground truth via face recognition. To do this, we have utilised a pre-trained convolutional neural network based on the VGG-face model for feature extraction, and we then use well-known classifiers to compare the features. We have utilised two datasets, namely the FEI and the Morph II, to test, verify and validate our approach. Our experimental results do suggest that the proposed approach achieves accuracy, efficiency and possess flexibility when it comes to facial age progression or regression.

The human visual system must quickly and accurately deploy task-and-object-specific processing to successfully navigate the environment, which suggests several interesting research questions: What is the nature of these strategies? Are they flexible? To what extent is this behaviour optimal given the natural statistics of the environment? In this thesis, I explored these questions using human faces, a complex and dynamic source of socially relevant information that we encounter throughout our lives. Specifically, I conducted several experiments examining the role of horizontally-oriented spatial frequency components in face identification. In Chapter 2, I use computational modelling to demonstrate that the structure conveyed by these components is maximally diagnostic for face identity, and show that selective processing of this structure predicts both face identification performance and the face inversion effect. In Chapter 3, I quantify the bandwidth utilized by human observers and relate this sampling strategy to the information structure of face stimuli. In Chapter 4, I show that the selective sampling described in Chapters 2 and 3 is driven by information from the eyes. Finally, in Chapter 5, I show that the impaired horizontal selectivity associated with face inversion is enhanced by practice identifying inverted faces. Together, these experiments characterize a stimulus with differentially diagnostic information sources that, through experience, becomes selectively processed in a manner associated with task performance. These results contribute to our understanding of expert object processing and may have implications for observers experiencing face perception deficits. / Thesis / Doctor of Philosophy (PhD)

Faces are among the most complex stimuli that the human visual system processes. Growing commercial interest in face recognition is encouraging, but it also turns out to be a challenging endeavour. These challenges arise when the situations are complex and cause varied facial appearance due to e.g., occlusion, low-resolution, and ageing. The problem of computer-based face recognition using partial facial data is still largely an unexplored area of research and how does computer interpret various parts of the face. Another challenge is age progression and regression, which is considered to be the most revealing topic for understanding the human face changes during life. In this research, the various computational face recognition models are investigated to overcome the challenges posed by ageing and occlusions/partial faces. For partial face-based face recognition, a pre-trained VGGF model is employed for feature extraction and then followed by popular classifiers such as SVMs and Cosine Similarity CS for classification. In this framework, parts of faces such as eyes, nose, forehead, are used individually for training and testing. The results showing that there is an improvement in recognition in small parts, such as recognition rate in forehead enhanced form about 0% to nearly 35%, eyes from about 22% to approximately 65%. In the second framework, five sub-models were built based on Convolutional Neural Networks (CNNs) and those models are named Eyes-CNNs, Nose-CNNs, Mouth-CNNs, Forehead-CNNs, and combined EyesNose-CNNs. The experimental results illustrate a high recognition rate when it comes to small parts, for example, eyes increased up to about 90.83% and forehead reached about 44.5%. Furthermore, the challenge of face ageing is also approached by proposing an age-template based framework, generating an age-based face template for enhanced face generation and recognition. The results showing that generated new aged faces are more reliable comparing with state-of-the-art.

Objective 1: To examine the influence of maternal depression and/ or anxiety on infant, maternal and dyadic FFSF task performance Objective 2: To investigate the changes in infant and maternal FFSF task performance before and after Cognitive Behavioural Therapy (CBT) treatment of maternal depression / Background: Research suggests that postpartum depression (PPD) and postpartum anxiety (PPA) impact both mothers and their infants, leading to adverse behavioural outcomes across the lifespan. The face-to-face still-face (FFSF) task is a validated observational tool used to measure the quality of mother-infant interactions. This thesis aimed to investigate the differences in responses to the FFSF task between dyads consisting of mothers with PPD and/or PPA and healthy dyads. Another goal was to examine whether PPD treatment could improve mother and infant FFSF outcomes. Methods: A systematic search was performed in PubMed/MEDLINE, EMBASE, CINAHL, PsycINFO and Web of Science. Meta-analyses were conducted to examine the differences in infant, maternal and dyadic FFSF outcomes in mothers with PPD, PPA or comorbid PPD and PPA in comparison to healthy control dyads. Second, we examined whether group cognitive behavioural therapy (CBT) for PPD could help improve infant and maternal FFSF outcomes. A case-control design study was conducted with three different assessment points (i.e., pre-CBT treatment, immediately after CBT and three months post-CBT). Results: Meta-analyses suggested that the infants of mothers with PPD display lower levels of positive affect during the play and reunion phases compared to the infants of healthy non-depressed mothers. Also, mothers with PPD may engage less positively with their infants at the reunion phase, and mother-infant dyads affected by PPD show less positive interactive matching during the play phase compared to healthy control dyads. Finally, object/environment engagement was higher in infants of PPA mothers compared to healthy controls at still-face. Conclusion: The results suggest that mothers with PPD and/or PPD (and their infants) may exhibit different interaction patterns compared to healthy dyads. Also, it appears that the benefits of CBT for maternal PPD may extend to their infants through reductions in maladaptive infant withdrawn behaviours to normal, healthy levels. / Thesis / Master of Science (MSc) / Maternal postpartum depression (PPD) and postpartum anxiety (PPA) are the most common mental health complications of birth. Apart from unfavourable effects PPD and PPA have on mothers, it may also impact the mother-infant relationship, leading to adverse infant outcomes. Given the relatively high prevalence of maternal PPD, PPA, and comorbid PPD and PPA, this thesis aimed to examine the differences in how mothers suffering from PPD and/or PPA and their infants coordinate their behaviour, in comparison to healthy mothers and their infants using a validated observational task (face-to-face still-face [FFSF] task). Another goal of this thesis was to investigate whether the benefits of maternal treatment for PPD with cognitive behavioural therapy may extend to infants and improve mother, as well as infant behaviour. These investigations may provide new insights on how maternal PPD and/or PPA affects mother-infant interactions, and consequently, infant socio-emotional development.

Human face recognition is an important area in the field of biometrics. It has been an active area of research for several decades, but still remains a challenging problem because of the complexity of the human face. In this thesis we describe fully automatic solutions that can locate faces and then perform identification and verification. We present a solution for face localisation using eye locations. We derive an efficient representation for the decision hyperplane of linear and nonlinear Support Vector Machines (SVMs). For this we introduce the novel concept of $\rho$ and $\eta$ prototypes. The standard formulation for the decision hyperplane is reformulated and expressed in terms of the two prototypes. Different kernels are treated separately to achieve further classification efficiency and to facilitate its adaptation to operate with the fast Fourier transform to achieve fast eye detection. Using the eye locations, we extract and normalise the face for size and in-plane rotations. Our method produces a more efficient representation of the SVM decision hyperplane than the well-known reduced set methods. As a result, our eye detection subsystem is faster and more accurate. The use of fractals and fractal image coding for object recognition has been proposed and used by others. Fractal codes have been used as features for recognition, but we need to take into account the distance between codes, and to ensure the continuity of the parameters of the code. We use a method based on fractal image coding for recognition, which we call the Fractal Neighbour Distance (FND). The FND relies on the Euclidean metric and the uniqueness of the attractor of a fractal code. An advantage of using the FND over fractal codes as features is that we do not have to worry about the uniqueness of, and distance between, codes. We only require the uniqueness of the attractor, which is already an implied property of a properly generated fractal code. Similar methods to the FND have been proposed by others, but what distinguishes our work from the rest is that we investigate the FND in greater detail and use our findings to improve the recognition rate. Our investigations reveal that the FND has some inherent invariance to translation, scale, rotation and changes to illumination. These invariances are image dependent and are affected by fractal encoding parameters. The parameters that have the greatest effect on recognition accuracy are the contrast scaling factor, luminance shift factor and the type of range block partitioning. The contrast scaling factor affect the convergence and eventual convergence rate of a fractal decoding process. We propose a novel method of controlling the convergence rate by altering the contrast scaling factor in a controlled manner, which has not been possible before. This helped us improve the recognition rate because under certain conditions better results are achievable from using a slower rate of convergence. We also investigate the effects of varying the luminance shift factor, and examine three different types of range block partitioning schemes. They are Quad-tree, HV and uniform partitioning. We performed experiments using various face datasets, and the results show that our method indeed performs better than many accepted methods such as eigenfaces. The experiments also show that the FND based classifier increases the separation between classes. The standard FND is further improved by incorporating the use of localised weights. A local search algorithm is introduced to find a best matching local feature using this locally weighted FND. The scores from a set of these locally weighted FND operations are then combined to obtain a global score, which is used as a measure of the similarity between two face images. Each local FND operation possesses the distortion invariant properties described above. Combined with the search procedure, the method has the potential to be invariant to a larger class of non-linear distortions. We also present a set of locally weighted FNDs that concentrate around the upper part of the face encompassing the eyes and nose. This design was motivated by the fact that the region around the eyes has more information for discrimination. Better performance is achieved by using different sets of weights for identification and verification. For facial verification, performance is further improved by using normalised scores and client specific thresholding. In this case, our results are competitive with current state-of-the-art methods, and in some cases outperform all those to which they were compared. For facial identification, under some conditions the weighted FND performs better than the standard FND. However, the weighted FND still has its short comings when some datasets are used, where its performance is not much better than the standard FND. To alleviate this problem we introduce a voting scheme that operates with normalised versions of the weighted FND. Although there are no improvements at lower matching ranks using this method, there are significant improvements for larger matching ranks. Our methods offer advantages over some well-accepted approaches such as eigenfaces, neural networks and those that use statistical learning theory. Some of the advantages are: new faces can be enrolled without re-training involving the whole database; faces can be removed from the database without the need for re-training; there are inherent invariances to face distortions; it is relatively simple to implement; and it is not model-based so there are no model parameters that need to be tweaked.

<p>The main purpose of this study is to establish a socio-pragmatic categorization of politeness and impoliteness activities in informal interactions. In doing this, we describe the communicative strategies related to (im) politeness phenomena and how they are used to produce certain <i>social effects</i> in face-to-face interaction through the ongoing negotiation of participants’ <i>face </i>(Goffman, 1967). This study is based on informal conversations extracted from a <i>corpus </i>of spoken Spanish gathered in the metropolitan area of Valencia, Spain (Briz and Val.Es.Co. Group, 2002). Focusing on methodology, this study combines a qualitative method inspired in CA with a DA interpretative approach that analyzes communicative acts (Allwood 1995; Bravo, e. p.1). <i>Face</i> contents such as <i>autonomy </i>and<i> affiliative face, role face, group</i> and <i>individual face, </i>are a resource for analyzing what happens during interaction along with the resulting interpersonal effects. The integration of the analysis of context, which includes the co-text, the situational context and the socio-cultural context (cultural settings and shared assumptions), is equally important in this study. The empirical analysis of both the conversations and a questionnaire on impoliteness bring us to propose a series of categories of (im) politeness. The categories are as follow: Strategic Politeness (within this category we find <i>attenuating politeness </i>and<i> reparatory politeness</i>), Enhancing Politeness, Group Politeness, Ritual Politeness (here we differentiate between meeting situations and visit situations) and Discursive Politeness (we divide this category into <i>conventional </i>and <i>thematic</i>). Concerning Impoliteness, we find situations in informal conversation in which impoliteness is expected (<i>normative impoliteness</i>) and when threatening acts (reproaches, criticism, etc.) do not imply directly, <i>per se</i>, a negative personal effect. We next find two types of impoliteness: one produced by threats to the <i>face </i>of the speaker which are neither mitigated nor amended and the other caused by a break from the normal rules of politeness. </p>

The main purpose of this study is to establish a socio-pragmatic categorization of politeness and impoliteness activities in informal interactions. In doing this, we describe the communicative strategies related to (im) politeness phenomena and how they are used to produce certain social effects in face-to-face interaction through the ongoing negotiation of participants’ face (Goffman, 1967). This study is based on informal conversations extracted from a corpus of spoken Spanish gathered in the metropolitan area of Valencia, Spain (Briz and Val.Es.Co. Group, 2002). Focusing on methodology, this study combines a qualitative method inspired in CA with a DA interpretative approach that analyzes communicative acts (Allwood 1995; Bravo, e. p.1). Face contents such as autonomy and affiliative face, role face, group and individual face, are a resource for analyzing what happens during interaction along with the resulting interpersonal effects. The integration of the analysis of context, which includes the co-text, the situational context and the socio-cultural context (cultural settings and shared assumptions), is equally important in this study. The empirical analysis of both the conversations and a questionnaire on impoliteness bring us to propose a series of categories of (im) politeness. The categories are as follow: Strategic Politeness (within this category we find attenuating politeness and reparatory politeness), Enhancing Politeness, Group Politeness, Ritual Politeness (here we differentiate between meeting situations and visit situations) and Discursive Politeness (we divide this category into conventional and thematic). Concerning Impoliteness, we find situations in informal conversation in which impoliteness is expected (normative impoliteness) and when threatening acts (reproaches, criticism, etc.) do not imply directly, per se, a negative personal effect. We next find two types of impoliteness: one produced by threats to the face of the speaker which are neither mitigated nor amended and the other caused by a break from the normal rules of politeness.

Human face recognition is an important area in the field of biometrics. It has been an active area of research for several decades, but still remains a challenging problem because of the complexity of the human face. In this thesis we describe fully automatic solutions that can locate faces and then perform identification and verification. We present a solution for face localisation using eye locations. We derive an efficient representation for the decision hyperplane of linear and nonlinear Support Vector Machines (SVMs). For this we introduce the novel concept of $\rho$ and $\eta$ prototypes. The standard formulation for the decision hyperplane is reformulated and expressed in terms of the two prototypes. Different kernels are treated separately to achieve further classification efficiency and to facilitate its adaptation to operate with the fast Fourier transform to achieve fast eye detection. Using the eye locations, we extract and normalise the face for size and in-plane rotations. Our method produces a more efficient representation of the SVM decision hyperplane than the well-known reduced set methods. As a result, our eye detection subsystem is faster and more accurate. The use of fractals and fractal image coding for object recognition has been proposed and used by others. Fractal codes have been used as features for recognition, but we need to take into account the distance between codes, and to ensure the continuity of the parameters of the code. We use a method based on fractal image coding for recognition, which we call the Fractal Neighbour Distance (FND). The FND relies on the Euclidean metric and the uniqueness of the attractor of a fractal code. An advantage of using the FND over fractal codes as features is that we do not have to worry about the uniqueness of, and distance between, codes. We only require the uniqueness of the attractor, which is already an implied property of a properly generated fractal code. Similar methods to the FND have been proposed by others, but what distinguishes our work from the rest is that we investigate the FND in greater detail and use our findings to improve the recognition rate. Our investigations reveal that the FND has some inherent invariance to translation, scale, rotation and changes to illumination. These invariances are image dependent and are affected by fractal encoding parameters. The parameters that have the greatest effect on recognition accuracy are the contrast scaling factor, luminance shift factor and the type of range block partitioning. The contrast scaling factor affect the convergence and eventual convergence rate of a fractal decoding process. We propose a novel method of controlling the convergence rate by altering the contrast scaling factor in a controlled manner, which has not been possible before. This helped us improve the recognition rate because under certain conditions better results are achievable from using a slower rate of convergence. We also investigate the effects of varying the luminance shift factor, and examine three different types of range block partitioning schemes. They are Quad-tree, HV and uniform partitioning. We performed experiments using various face datasets, and the results show that our method indeed performs better than many accepted methods such as eigenfaces. The experiments also show that the FND based classifier increases the separation between classes. The standard FND is further improved by incorporating the use of localised weights. A local search algorithm is introduced to find a best matching local feature using this locally weighted FND. The scores from a set of these locally weighted FND operations are then combined to obtain a global score, which is used as a measure of the similarity between two face images. Each local FND operation possesses the distortion invariant properties described above. Combined with the search procedure, the method has the potential to be invariant to a larger class of non-linear distortions. We also present a set of locally weighted FNDs that concentrate around the upper part of the face encompassing the eyes and nose. This design was motivated by the fact that the region around the eyes has more information for discrimination. Better performance is achieved by using different sets of weights for identification and verification. For facial verification, performance is further improved by using normalised scores and client specific thresholding. In this case, our results are competitive with current state-of-the-art methods, and in some cases outperform all those to which they were compared. For facial identification, under some conditions the weighted FND performs better than the standard FND. However, the weighted FND still has its short comings when some datasets are used, where its performance is not much better than the standard FND. To alleviate this problem we introduce a voting scheme that operates with normalised versions of the weighted FND. Although there are no improvements at lower matching ranks using this method, there are significant improvements for larger matching ranks. Our methods offer advantages over some well-accepted approaches such as eigenfaces, neural networks and those that use statistical learning theory. Some of the advantages are: new faces can be enrolled without re-training involving the whole database; faces can be removed from the database without the need for re-training; there are inherent invariances to face distortions; it is relatively simple to implement; and it is not model-based so there are no model parameters that need to be tweaked.