When people interact with each other, they not only listen to what the other says, they react to facial expressions, gaze direction, and head movement. Human-computer interaction would be enhanced in a friendly and non-intrusive way if computers could understand and respond to users’ body language in the same way. This thesis aims to investigate new methods for human computer interaction by combining information from the body language of the head to recognize the emotional and cognitive states. We concentrated on the integration of facial expression, eye gaze and head movement using soft computing techniques. The whole procedure is done in two-stage. The first stage focuses on the extraction of explicit information from the modalities of facial expression, head movement, and eye gaze. In the second stage, all these information are fused by soft computing techniques to infer the implicit emotional states. In this thesis, the frequency of head movement (high frequency movement or low frequency movement) is taken into consideration as well as head nods and head shakes. A very high frequency head movement may show much more arousal and active property than the low frequency head movement which differs on the emotion dimensional space. The head movement frequency is acquired by analyzing the tracking results of the coordinates from the detected nostril points. Eye gaze also plays an important role in emotion detection. An eye gaze detector was proposed to analyze whether the subject's gaze direction was direct or averted. We proposed a geometrical relationship of human organs between nostrils and two pupils to achieve this task. Four parameters are defined according to the changes in angles and the changes in the proportion of length of the four feature points to distinguish avert gaze from direct gaze. The sum of these parameters is considered as an evaluation parameter that can be analyzed to quantify gaze level. The multimodal fusion is done by hybridizing the decision level fusion and the soft computing techniques for classification. This could avoid the disadvantages of the decision level fusion technique, while retaining its advantages of adaptation and flexibility. We introduced fuzzification strategies which can successfully quantify the extracted parameters of each modality into a fuzzified value between 0 and 1. These fuzzified values are the inputs for the fuzzy inference systems which map the fuzzy values into emotional states.

Non-suicidal self-injury (NSSI) has been regarded as a dysfunctional coping strategy for managing intensely difficult feelings and is a growing area of concern in clinical and non-clinical populations alike. Individuals who engage in NSSI often report significant interpersonal difficulties, with studies showing that they have impaired social interactions. Attending to the emotional expressions of others is important for appropriate social functioning, and subtly mimicking the expressions of others is an unconscious behaviour that encourages empathy and interpersonal bonding. Differences in emotion recognition and reflection can impact social behaviour, yet little research has assessed how individuals with a history of NSSI (HNSSI) process facial expressions of emotion. In this thesis, the results of three studies designed to investigate potential differences in emotion recognition and emotion mimicry in individuals with a history of NSSI compared to controls are reported. Results from Study 1 revealed that when presented with colourful, dynamic morphing stimuli showing emotional expressions, HNSSI participants were able to correctly categorize negative expressions like anger, disgust, sadness, and the ambiguous emotion of surprise at a lower stimulus intensity compared to controls; They also correctly categorized fear with greater accuracy, though at the same intensity as controls. However, in Study 2, when static, greyscale images were obscured with varying levels of fractal noise, HNSSI individuals did not show superior signal-proportion thresholds. These results may suggest that higher-order elements of visual and cognitive processing are evoked by more realistic social stimuli. In the third study, HNSSI participants were found to produce significantly less electrical activity in the corrugator muscle in response to viewing angry stimuli, and significantly less of an expected relaxation in muscle activity in response to viewing happy stimuli. Complementing these results, it was found that endorsing social influence as a motivator for engaging in NSSI was associated with less mimicry, whereas endorsing emotion regulation as a motivator was associated with greater incongruent muscle response when viewing happy faces. These observed differences in facial mimicry between HNSSI and controls may be related to some of the observed relationship difficulties experienced by this group. Results from this research may help us to better understand NSSI behaviour, as well as help to inform and further develop therapies intended to address it.

No description available.

No description available.

Objective: The overall aim of this study was to investigate whether maternal emotion recognition of infant faces in a facial morphing task differed by maternal attachment style, and if this was moderated by a secure attachment prime, such that it would ameliorate the effects of maternal attachment insecurity. Method: 87 mothers of children aged 0-18 months completed measures of global and mother-specific trait attachment, post-natal depression, mood and state attachment alongside 2 sessions of an emotion recognition task. This task was made up of short movies created from photographs of infant faces, changing from neutral to either happy or sad. It was designed to assess sensitivity (accuracy of responses and intensity of emotion required to recognize the emotion) to changes in emotions expressed in the faces of infants. Participants also underwent a prime manipulation that was either attachment-based (experimental group) or neutral (control group). Results: There were no significant effects for global attachment scores (i.e., avoidant, anxious). However, there was a significant interaction effect of condition x maternal avoidant attachment for accuracy of recognition of happy infant faces. Explication of this interaction yielded an unexpected finding: participants reporting avoidant attachment relationships with their own mothers were less accurate in recognizing happy infant faces following the attachment prime than participants with maternal avoidant attachment in the control condition. Conclusions: Future research directions suggest ways to improve strength of effects and variability in attachment insecurity. Clinical implications of the study center on the preliminary evidence presented that supports carefully selected and executed interventions for mothers with attachment problems.

Objectives: This study sought to determine whether a computerised cognitive bias modification programme could be effective within a waiting-room setting for parents accompanying their children to CAMHS appointments. The primary objectives were to determine whether detectable changes to participants' emotion recognition could be observed in this setting, and whether this approach would be acceptable to the population. Secondary measures investigated whether the programme would lead to changes in participants' affect or changes in parents' appraisals of difficulties with children. Methods: A computerised emotion recognition training task was delivered to all participants during four weekly sessions. Participants in the experimental condition (n=17) received feedback aiming to shift their detection of positive facial emotions, while those in the control condition (n=14) received feedback which was not designed to elicit any shift in emotion detection. Results: Positive shifts in emotion recognition were observed in the experimental group, although no changes were observed in secondary measures in either control or experimental groups. Qualitative data indicated that the programme was acceptable and appropriately constructed. Conclusion: This study demonstrates that cognitive bias modification is possible within a waiting-room setting, although the extent to which this can lead to clinically significant improvements in mood or relationships remains uncertain. This work has implications for emotion recognition interventions for clinical populations known to present with negative emotional biases (e.g. anxiety and depression) and represents an important first research step towards developing interventions to improve parent-child relationships.

Objectives: There is evidence that childhood traumatic brain injury (TBI) is associated with increased risk of offending and violent crime. This study aimed to explore associations between TBI in a group of delinquent and vulnerable young people (VYP) at risk of offending, and facial emotion recognition (FER) abilities, inhibition control (Stop-IT) and self-reported reactive-proactive aggression (RPQ). Methods: There were two studies. The first study used a cross sectional between group design to compare 45 VYP (with and without TBI) and a control group of 59 students on FER task measuring emotion recognition accuracy of six basic emotions. The second study examined differences between TBI and non-TBI groups in the VYP sample (N=21) on a Stop-IT task, FER accuracy and self-reported reactive-proactive aggression. Results: A history of TBI was reported by 60% of the VYP group (48.9% with loss of consciousness [LoC]), whereas 30% of the control group reported a history of TBI (25.4% with LoC). The VYP group (with and without TBI) demonstrated a similar pattern of reduced overall FER accuracy that was significantly different to the control group. Compared to the control group, The VYP groups (with and without TBI) were less accurate on recognising anger, disgust, sadness and surprise, but not happy and fear. There were no significant differences between the TBI- and non-TBI groups. The second study did not find any significant differences between the TBI and non-TBI groups on overall FER accuracy, Stop-IT performance, and RPQ scores. There were also no significant associations between these measures. Conclusions: Future research requires larger samples that enable investigating the association between different severity of TBI, FER and inhibition control ability. Better and more youth-friendly measures are also needed.

Understanding others’ emotions and false beliefs, known as Theory of Mind (ToM), and to recognise and produce facial expressions of emotion has been linked to social competence. Deaf children born to hearing parents have commonly shown a deficit, or at best a delay in ToM. The emotion processing skills of deaf children are less clear. The main aims of this thesis were to clarify the ability of emotion recognition in deaf children, and to provide the first investigation in emotion production. While deaf children were poorer than hearing controls at recognising expressions of emotion in cartoon faces, a similar pattern was found in both groups’ recognition of real human faces of the six basic emotions (happiness, sadness, anger, fear, disgust and surprise). For deaf children, emotion recognition was better in dynamic rather than static, and intense rather than subtle, displays of emotion. With the exception of disgust, no differences in individual emotions were found, suggesting that the use of ecologically valid dynamic real faces facilitates deaf children’s emotion recognition. Deaf children’s ability to produce the six basic emotions was compared to hearing children by videoing voluntary encodings of facial expression elicited via verbal labels and emotion signed stories, and the imitation of dynamic displays of real facial expressions of emotion. With the exception of a poorer performance in imitation and the verbally elicited production of disgust, deaf children were consistently rated by human judges overall as producing more recognisable and intense expressions, suggesting that clarity and expressiveness may be important to deaf individuals’ emotion display rules. In line with previous studies, results showed a delay in passing the first and second order belief tasks in comparison to age matched controls, but not in comparison to a group of ‘age appropriate’ hearing control children. These findings encouragingly suggest that while deaf children of hearing parents show a delay in ToM and understanding disgust, emotion processing skills follow a broadly similar pattern of development to hearing control children. Language experience is implicated in difficulties faced in social and emotion cognition, with reduced opportunities to discuss more complex emotional and mental states.

O trabalho pioneiro de Darwin, em 1872, constituiu uma das primeiras fontes de informação sobre as emoções dentro da perspectiva evolucionista. Hoje podemos contar com equipamentos de medidas mais sofisticados que nos permitem revisitar alguma das questões básicas por ele formuladas. A presente tese vem contribuir para uma melhor compreensão do reconhecimento de emoções e teoria da mente em crianças com TEA, apresentando analises que possam levar a sugestões de intervenções clínicas futuras. Embora ainda haja inconsistência entre os estudos quanto a qual emoção está mais prejudicada, tem sido relatadas dificuldades na identificação das emoções de medo, raiva, nojo e surpresa. Usamos um eye-tracker com o objetivo de esclarecer como os participantes olham para estímulos. Os participantes foram 40 meninos com TEA e com desenvolvimento típico, equiparados quanto a idade (6 a 14 anos). Foram avaliados para classificação do QI e para o grau de autismo. Os estímulos visuais foram retirados dos subteste de Reconhecimento de Emoções e Teoria da Mente do NEPSY II e apresentados em um equipamento de rastreamento de olhar, tendo sido definidas áreas de interesse (AI) do rosto. Para cada imagem, foi computado o tempo de fixação (em segundos) em cada AI. Os participantes também realizaram as tarefas do subteste, com o objetivo de verificar o acerto e o erro em cada estímulo. Em nossas análises pudemos verificar que a idade e o QI dos participantes influenciaram o desempenho na tarefa de reconhecimento de emoções e não influenciaram na tarefa de ToM. As crianças com TEA apresentaram um desempenho prejudicado quando comparado com as crianças com DT em ambos subtestes. Os participantes com TEA erraram mais as tarefas com emoções de valência negativa, como tristeza e raiva. A classificação do grau de TEA, não teve influência no desempenho dos participantes. O que parece ocorrer é que pertencer ao grupo clínico TEA seja uma condição para ter um desempenho pior quando comparado com o grupo de DT. Analisando o rastreamento de olhar, constatamos que o grupo DT permaneceu menos tempo fixando o olhar na AI dos olhos e mais tempo na AI da boca. Como o tempo total de fixação para os dois grupos não teve diferença estatística no subteste de reconhecimento de emoção, podemos sugerir que as crianças com TEA apresentam um padrão de olhar que chamamos de \"difuso\" enquanto que as crianças com DT apresentam um padrão de olhar que chamamos de \"focado\" nas regiões da face relevantes para o reconhecimento de emoções. Parece que na tarefa de ToM, o tempo de fixação em olhos e boca não foi o determinante para o erro na resposta e sim o olhar difuso na cena que deveria ser interpretada. Quando avaliamos os resultados qualitativos do rastreamento do olhar podemos pensar na importância de intervenções que possam ajudar crianças com TEA a melhorar seu desempenho em identificar as expressões faciais que auxiliam na decodificação da emoção e na interpretação de estados mentais / Darwin\'s pioneering work in 1872 was one of the earliest sources of information about emotions within the evolutionary perspective. Today we have more sophisticated equipment that allows us to revisit some of the basic questions that he formulates. The present thesis contributed to a better understanding of the emotion recognition and theory of mind in children with ASD, presenting analyzes that may lead to suggestions of future clinical interventions. Although there is still inconsistency among studies as to which emotion is most impaired, difficulties in identifying the emotions of fear, anger, disgust and surprise have been reported. We use an eye-tracker to clarify how participants look at stimuli. The participants were 40 boys with ASD and typical development, matched for age (6 to 14 years). They were evaluated for IQ and for the level of autism. The visual stimuli were from the Emotional Recognition and Theory of Mind subtests of NEPSY II and presented with an eye-tracking device, with areas of interest (AI) defined. For each image, the fixation duration (in seconds) in each AI was computed. The participants also performed the tasks of the subtest, in order to verify the right and wrong answers in each stimulus. In our analyzes we could verify that the participants\' age and IQ influenced the performance in the task of emotion recognition and did not influence the task of ToM. Children with ASD had an impaired performance when compared to children with TD in both subtests. Participants with ASD had a low score in tasks with negative valence emotions, such as sadness and anger. The classification of the ASD level had no influence on the participants\' performance. What seems to occur is that beeing on the ASD clinical group is a condition to had a worse perform when compared to the TD group. Analyzing the eye tracking, we found that the TD group spent less time fixing the eyes in the AI of the eyes and longer in the AI of the mouth. As the total fixation time for the two groups did not have a statistical difference in the emotion recognition subtest, we can suggest that children with ASD have a look pattern that we call \"diffuse\", while children with TD have a look pattern which we call \"focused\" on regions of the face relevant to the recognition of emotions. It seems that in the ToM task, the fixation time in the eyes and mouth was not the determinant for the error in the response, but the diffuse look in the scene that was to be interpreted. When we evaluate the qualitative results of eye tracking we can think of the importance of interventions that can help children with ASD improve their performance in identifying the facial expressions that aid in the decoding of emotion and the interpretation of mental states

Objective: Neuropsychology models of anorexia nervosa (AN) propose that cognitive difficulties including poor Emotion Recognition (ER) and set-shifting ability may be central to the development and maintenance of eating pathology. This study aimed to test the central positions of such models by assessing specific ER difficulties in AN as well as the relationship between ER deficits and set-shifting performance. Methods: Fifty-one women were assessed (25 with AN; M = 28.20 SD = 8.69 and 26 control M = 21.27 SD = 5.10) on a novel measure of ER, a set-shifting test and self-report questionnaires concerning co-morbid factors. Results: The data did not reveal a global difference in ER or set-shifting performance between groups. Specific hypotheses of ER deficits in AN were also not met as performance on individual emotions was comparable between groups. There was an unexpected negative correlation between disgust recognition and set-shifting performance, however, this was only significant across the whole sample. ER performance was not related with any confounding factors. Conclusions: Despite an abundance of research supporting the position of social cognitive difficulties in AN, the current study failed to find global or specific deficits in ER in the present sample. Similarly, ER performance was not related to set-shifting as proposed by neuropsychological models of AN aetiology. Possible explanations for a lack of difference observed using this novel ER task are explored and future directions for evaluating ER in AN are discussed.