Objective: Research has demonstrated deficits in socio-emotional processing following childhood traumatic brain injury (TBI; Tonks et al., 2009a). However, it is not known whether a link exists between socio-emotional processing, TBI and offending. Drawing on Ochsner’s (2008) socio-emotional processing model, the current study aimed to investigate facial emotion recognition accuracy and bias in young offenders with TBI. Setting: Research was conducted across three youth offender services. Participants: Thirty seven participants completed the study. Thirteen participants reported a high dosage of TBI. Design: The study had a cross sectional within and between subjects design. Main Measures: Penton-Voak and Munafo’s (2012) emotional recognition task was completed. Results: The results indicated that young offenders with a TBI were not significantly worse at facial emotion recognition compared to those with no TBI. Both groups showed a bias towards positive emotions. No between group differences were found for emotion bias. Conclusion: The findings did not support the use of Ochsner’s (2008) socio-emotional processing model for this population. Due to the small sample size, inadequate power and lack of non-offender control groups, it is not possible to draw any firm conclusions from the results of this study. Future research should aim to investigate whether there are any links between TBI, socio-emotional processing and offending.

The main topic of this thesis is emotional intelligence. The theoretical part is dedicated to the definition of emotional intelligence and the introduction to the problematic of this phenomenon. There is also a chapter dedicated to a review of emotional intelligence measures and critical assessment of the selected methods. Special focus is given to the approach of Mayer and Salovey. They defined emotional intelligence as the ability to understand, express, use and regulate emotions in self and others. On the base of their theory, they developed a method MSCEIT (Mayer Salovey Caruso Emotional Intelligence Test). The thesis is also reviewing a Bar-On's method and his approach. The purpose of the empirical part of the thesis is to validate the newly developed method TRE (Emotion Recognition Test) which was developed by a team from QED GROUP company. The data from this test and from the golden standard in measuring emotional intelligence - MSCEIT were compared by correlation. The result of the correlation between TRE and MSCEIT showed correlation on a significant level on the sample N = 65. The correlation between TRE and the total score of MSCEIT showed a satisfactory level of convergent validity (r = .249, p = .009). The highest correlation was found between TRE and MSCEIT dimension Using emotions...

abstract: While discrete emotions like joy, anger, disgust etc. are quite popular, continuous emotion dimensions like arousal and valence are gaining popularity within the research community due to an increase in the availability of datasets annotated with these emotions. Unlike the discrete emotions, continuous emotions allow modeling of subtle and complex affect dimensions but are difficult to predict. Dimension reduction techniques form the core of emotion recognition systems and help create a new feature space that is more helpful in predicting emotions. But these techniques do not necessarily guarantee a better predictive capability as most of them are unsupervised, especially in regression learning. In emotion recognition literature, supervised dimension reduction techniques have not been explored much and in this work a solution is provided through probabilistic topic models. Topic models provide a strong probabilistic framework to embed new learning paradigms and modalities. In this thesis, the graphical structure of Latent Dirichlet Allocation has been explored and new models tuned to emotion recognition and change detection have been built. In this work, it has been shown that the double mixture structure of topic models helps 1) to visualize feature patterns, and 2) to project features onto a topic simplex that is more predictive of human emotions, when compared to popular techniques like PCA and KernelPCA. Traditionally, topic models have been used on quantized features but in this work, a continuous topic model called the Dirichlet Gaussian Mixture model has been proposed. Evaluation of DGMM has shown that while modeling videos, performance of LDA models can be replicated even without quantizing the features. Until now, topic models have not been explored in a supervised context of video analysis and thus a Regularized supervised topic model (RSLDA) that models video and audio features is introduced. RSLDA learning algorithm performs both dimension reduction and regularized linear regression simultaneously, and has outperformed supervised dimension reduction techniques like SPCA and Correlation based feature selection algorithms. In a first of its kind, two new topic models, Adaptive temporal topic model (ATTM) and SLDA for change detection (SLDACD) have been developed for predicting concept drift in time series data. These models do not assume independence of consecutive frames and outperform traditional topic models in detecting local and global changes respectively. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2015

Dans le vieillissement « sain », la préférence pour les stimuli positifs augmente par rapport aux stimuli négatifs. Ce phénomène est appelé « effet de positivité » et peut être observé au niveau comportemental et cérébral. L'objectif principal de cette thèse a été de caractériser les effets de l'âge sur les traitements émotionnels afin d'améliorer notre compréhension des effets de positivité. L'objectif sous-jacent a été d'évaluer dans quelles conditions ces effets peuvent conduire à une plus grande vulnérabilité des personnes âgées face à des situations menaçantes. Une première étude en électroencéphalographie a révélé que l'engagement attentionnel pour des scènes naturelles négatives diminue avec l'âge quel que soit leur niveau d'activation dans une tâche de catégorisation affective. A l'inverse, ce dernier reste inchangé pour les situations positives, conduisant à une réduction des biais de négativité. Une deuxième étude en électroencéphalographie, dont le paradigme était similaire à la première étude, a mis en évidence que les biais de négativité restent préservés avec l'âge lorsque l'évaluation des scènes s'effectue sur la dimension de « tendance à l'action ». Une troisième étude révèle que l'attention volontaire sur les situations d'intérêt des personnes âgées (positives) et sur les processus d'évaluation modulés par l'âge est nécessaire à l'émergence des effets de positivité. Parallèlement à ces travaux, une méthodologie innovante est proposée pour la classification d'états émotionnels des personnes jeunes et âgées sur la base de leurs signaux électroencéphalographiques. Nous avons obtenu des résultats encourageants qui suggèrent la possibilité cette méthode pour implémenter des interfaces cerveau-machine pour protéger les personnes âgées d'une éventuelle vulnérabilité en raison des effets de positivité. L'ensemble de ces travaux suggèrent que les effets de positivité sont les conséquences de changements sur le plan motivationnel de l'individu âgé, touchant principalement les processus d'évaluation émotionnel. La personne âgée régulerait ses émotions et diminuerait l'impact des émotions négatives lorsque d'autres motivations plus prioritaires sont absentes. / With aging, the preference for positive stimuli increases compared to negative stimuli. This is called “positivity effect” and it may be observed in both behavior and brain activity. The main goal of this work was to characterized age effects on emotional processing to improve our understood of this positivity effect. The second goal was to evaluate in which conditions these effects could make older people more vulnerable when they are confronted to threatening situations. A first EEG study revealed that the attentional engagement decreased with age for negative stimuli, regardless of their activation level, in an affective categorization task. Conversely, the processing of positive stimuli was preserved with age and, consequently, a reduction of the negativity bias was observed. In a second EEG study, using a similar paradigm to study 1 with the exception of the task which was an “action tendency task”, we observed a preservation of the negativity bias. A third study revealed that the voluntary attention on interest situations for aged adults (positive) and on appraisal process modulated with age was requisite to observe positivity effects. Parallel to this work, a new method was proposed to recognize and classify emotional states based on EEG signals. We obtained encouraging results which suggest the possibility to use this method to elaborate brain-computer interfaces to protect old people against a potential vulnerability due to positivity effect. Taken together, these results demonstrate that positivity effect is due to motivational shifts with age. Older people would be motivated to increase their well-being and would regulate their emotions by reducing the impact of negative stimuli, provided no other more important motivations are absent.

Data obtained in laboratory settings is a valid but resource-demanding approach. Moreover, although aspects of both metacognition and metaemotion have been proposed to be important for socioemotional functioning, such associations have rarely been studied previously. This study aimed to examine the feasibility of a multimodal online-assessment of emotion recognition ability, and to investigate its associations with metacognition and metaemotion. The sample consisted of 106 students from three Swedish universities. The online-survey included a multimodal emotion recognition test (ERAM) with added trial-by-trial confidence judgments (to measure metacognition) and questionnaires related to metaemotion. Online-data showed great consistency with previous data collected in lab. Well-calibrated adults had higher emotion recognition accuracy than under-confident adults. Higher levels of negative metaemotions were associated with higher emotion recognition accuracy. In conclusion, online-assessments of emotional abilities may be a useful approach. Further research is required to understand relationships between metacognition, metaemotion, and emotion recognition ability more fully.

Context. Foreign Accent Syndrome is a rare neurological disorder, where among other symptoms of the patient’s emotional speech is affected. As FAS is one of the mildest speech disorders, there has not been much research done on the cost-effective biomarkers which reflect recovery of competences speech. Objectives. In this pilot study, we implement the Emotional Temperature biomarker and check its validity for assessing the FAS. We compare the results of implemented biomarker with another biomarker based on the global distances for FAS and identify the better one. Methods. To reach the objective, the emotional speech data of two patients at different phases of the treatment are considered. After preprocessing, experiments are performed on various window sizes and the observed correctly classified instances in automatic recognition are used to calculate Emotional temperature. Further, we use the better biomarker for tracking the recovery in the patient’s speech. Results. The Emotional temperature of the patient is calculated and compared with the ground truth and with that of the other biomarker. The Emotional temperature is calculated to track the emergence of compensatory skills in speech. Conclusions. A biomarker based on the frame-view of speech signal has been implemented. The implementation has used the state of art feature set and thus is an unproved version of the classical Emotional Temperature. The biomarker has been used to automatically assess the recovery of two patients diagnosed with FAS. The biomarker has been compared against the global view biomarker and has advantages over it. It also has been compared to human evaluations and captures the same dynamics.

The beauty of affective computing is to make machine more emphatic to the user. Machines with the capability of emotion recognition can actually look inside the user’s head and act according to observed mental state. In this thesis project, we investigate different features set to build an emotion recognition system from electroencephalographic signals. We used pictures from International Affective Picture System to motivate three emotional states: positive valence (pleasant), neutral, negative valence (unpleasant) and also to induce three sets of binary states: positive valence, not positive valence; negative valence, not negative valence; and neutral, not neutral. This experiment was designed with a head cap with six electrodes at the front of the scalp which was used to record data from subjects. To solve the recognition task we developed a system based on Support Vector Machines (SVM) and extracted the features, some of them we got from literature study and some of them proposed by ourselves in order to rate the recognition of emotional states. With this system we were able to achieve an average recognition rate up to 54% for three emotional states and an average recognition rate up to 74% for the binary states, solely based on EEG signals.

In this thesis we present novel methods for constructing and fitting 2d models of shape and appearance which are used for analysing human faces. The first contribution builds on previous work on discriminative fitting strategies for active appearance models (AAMs) in which regression models are trained to predict the location of shapes based on texture samples. In particular, we investigate non-parametric regression methods including random forests and Gaussian processes which are used together with gradient-like features for shape model fitting. We then develop two training algorithms which combine such models into sequences, and systematically compare their performance to existing linear generative AAM algorithms. Inspired by the performance of the Gaussian process-based regression methods, we investigate a group of non-linear latent variable models known as Gaussian process latent variable models (GPLVM). We discuss how such models may be used to develop a generative active appearance model algorithm whose texture model component is non-linear, and show how this leads to lower-dimensional models which are capable of generating more natural-looking images of faces when compared to equivalent linear models. We conclude by describing a novel supervised non-linear latent variable model based on Gaussian processes which we apply to the problem of recognising emotions from facial expressions.

Automatic processing of emotion information through deep neural networks (DNN) can have great benefits (e.g., for human-machine interaction). Vice versa, machine learning can profit from concepts known from human information processing (e.g., visual attention). We employed a recurrent DNN incorporating a spatial attention mechanism for facial emotion recognition (FER) and compared the output of the network with results from human experiments. The attention mechanism enabled the network to select relevant face regions to achieve state-of-the-art performance on a FER database containing images from realistic settings. A visual search strategy showing some similarities with human saccading behavior emerged when the model’s perceptive capabilities were restricted. However, the model then failed to form a useful scene representation.

京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第22800号 / 情博第730号 / 新制||情||125(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)准教授 中澤 篤志, 教授 西野 恒, 教授 鹿島 久嗣 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DGAM