Emotions play a significant role in many human mental activities, including decision-making, motivation, and cognition. Various intelligent and expert systems can be empowered with emotionally intelligent capabilities, especially systems that interact with humans and mimic human behaviour. However, most current methods in affect recognition studies use intrusive, lab-based, and expensive tools which are unsuitable for real-world situations. Inspired by studies on keystrokes dynamics, this thesis investigates the effectiveness of diagnosing users’ affect through their typing behaviour in an educational context. To collect users’ typing patterns, a field study was conducted in which subjects used a dialogue-based tutoring system built by the researcher. Eighteen dialogue features associated with subjective and objective ratings for users’ emotions were collected. Several classification techniques were assessed in diagnosing users’ affect, including discrimination analysis, Bayesian analysis, decision trees, and neural networks. An artificial neural network approach was ultimately chosen as it yielded the highest accuracy compared with the other methods. To lower the error rate, a hierarchical classification was implemented to first classify user emotions based on their valence (positive or negative) and then perform a finer classification step to determining which emotions the user experienced (delighted, neutral, confused, bored, and frustrated). The hierarchical classifier was successfully able to diagnose users' emotional valence, while it was moderately able to classify users’ emotional states. The overall accuracy obtained from the hierarchical classifier significantly outperformed previous dialogue-based approaches and in line with some affective computing methods.

abstract: Affect signals what humans care about and is involved in rational decision-making and action selection. Many technologies may be improved by the capability to recognize human affect and to respond adaptively by appropriately modifying their operation. This capability, named affect-driven self-adaptation, benefits systems as diverse as learning environments, healthcare applications, and video games, and indeed has the potential to improve systems that interact intimately with users across all sectors of society. The main challenge is that existing approaches to advancing affect-driven self-adaptive systems typically limit their applicability by supporting the creation of one-of-a-kind systems with hard-wired affect recognition and self-adaptation capabilities, which are brittle, costly to change, and difficult to reuse. A solution to this limitation is to leverage the development of affect-driven self-adaptive systems with a manufacturing vision. This dissertation demonstrates how using a software product line paradigm can jumpstart the development of affect-driven self-adaptive systems with that manufacturing vision. Applying a software product line approach to the affect-driven self-adaptive domain provides a comprehensive, flexible and reusable infrastructure of components with mechanisms to monitor a user’s affect and his/her contextual interaction with a system, to detect opportunities for improvements, to select a course of action, and to effect changes. It also provides a domain-specific architecture and well-documented process guidelines, which facilitate an understanding of the organization of affect-driven self-adaptive systems and their implementation by systematically customizing the infrastructure to effectively address the particular requirements of specific systems. The software product line approach is evaluated by applying it in the development of learning environments and video games that demonstrate the significant potential of the solution, across diverse development scenarios and applications. The key contributions of this work include extending self-adaptive system modeling, implementing a reusable infrastructure, and leveraging the use of patterns to exploit the commonalities between systems in the affect-driven self-adaptation domain. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2016

Purpose: This thesis investigated whether perceived parental care and overprotection predicted accuracy of face emotion recognition in psychiatrically healthy youth. The study also examined whether child gender and having a parent with a history of anxiety moderated the relationship between parental bonding and facial emotion recognition, and whether behavioural inhibition mediated this relationship. Methods: The sample comprised 176 males and females aged 7-18 years. Participants completed the Parental Bonding Instrument, Childhood Self-Report of Inhibition, and the Ekman emotion recognition task. Results: Child gender and parental history of anxiety moderated the relationship between perceived parenting style and affect recognition. In boys, overprotection by father predicted deficits in recognizing fearful faces; in children with parental anxiety, low paternal care predicted deficits in recognizing angry faces; and in boys with parental anxiety, negative maternal bonding predicted deficits in recognizing expressions of surprise. Also, maternal overprotection predicted intensity of subjective anxiety while viewing expressions of surprise and happiness for all offspring, and behaviour inhibition mediated these relationships. Implications: The present study provides preliminary evidence that parental bonding interacts with risk group and gender in predicting accuracy of facial affect recognition in healthy youth. Further research is needed to confirm these findings and determine whether the interaction between gender, risk group and deficits in social cognition increase risk for developing pathological anxiety.

No description available.

No description available.

<p>The goal of a machine learning problem is to learn useful patterns from observations so that appropriate inference can be made from new observations as they become available. Based on whether labels are available for training data, a vast majority of the machine learning approaches can be broadly categorized into supervised or unsupervised learning approaches. In the context of supervised learning, when observations are available as labeled feature vectors, the learning process is a well-understood problem. However, for many applications, the standard supervised learning becomes complicated because the labels for observations are unavailable as labeled feature vectors. For example, in a ground penetrating radar (GPR) based landmine detection problem, the alarm locations are only known in 2D coordinates on the earth's surface but unknown for individual target depths. Typically, in order to apply computer vision techniques to the GPR data, it is convenient to represent the GPR data as a 2D image. Since a large portion of the image does not contain useful information pertaining to the target, the image is typically further subdivided into subimages along depth. These subimages at a particular alarm location can be considered as a set of observations, where the label is only available for the entire set but unavailable for individual observations along depth. In the absence of individual observation labels, for the purposes of training standard supervised learning approaches, observations both above and below the target are labeled as targets despite substantial differences in their characteristics. As a result, the label uncertainty with depth would complicate the parameter inference in the standard supervised learning approaches, potentially degrading their performance. In this work, we develop learning algorithms for three such specific scenarios where: (1) labels are only available for sets of independent and identically distributed (i.i.d.) observations, (2) labels are only available for sets of sequential observations, and (3) continuous correlated multiple labels are available for spatio-temporal observations. For each of these scenarios, we propose a modification in a traditional learning approach to improve its predictive accuracy. The first two algorithms are based on a set-based framework called as multiple instance learning (MIL) whereas the third algorithm is based on a structured output-associative regression (SOAR) framework. The MIL approaches are motivated by the landmine detection problem using GPR data, where the training data is typically available as labeled sets of observations or sets of sequences. The SOAR learning approach is instead motivated by the multi-dimensional human emotion label prediction problem using audio-visual data, where the training data is available in the form of multiple continuous correlated labels representing complex human emotions. In both of these applications, the unavailability of the training data as labeled featured vectors motivate developing new learning approaches that are more appropriate to model the data. </p><p>A large majority of the existing MIL approaches require computationally expensive parameter optimization, do not generalize well with time-series data, and are incapable of online learning. To overcome these limitations, for sets of observations, this work develops a nonparametric Bayesian approach to learning in MIL scenarios based on Dirichlet process mixture models. The nonparametric nature of the model and the use of non-informative priors remove the need to perform cross-validation based optimization while variational Bayesian inference allows for rapid parameter learning. The resulting approach is highly generalizable and also capable of online learning. For sets of sequences, this work integrates Hidden Markov models (HMMs) into an MIL framework and develops a new approach called the multiple instance hidden Markov model. The model parameters are inferred using variational Bayes, making the model tractable and computationally efficient. The resulting approach is highly generalizable and also capable of online learning. Similarly, most of the existing approaches developed for modeling multiple continuous correlated emotion labels do not model the spatio-temporal correlation among the emotion labels. Few approaches that do model the correlation fail to predict the multiple emotion labels simultaneously, resulting in latency during testing, and potentially compromising the effectiveness of implementing the approach in real-time scenario. This work integrates the output-associative relevance vector machine (OARVM) approach with the multivariate relevance vector machine (MVRVM) approach to simultaneously predict multiple emotion labels. The resulting approach performs competitively with the existing approaches while reducing the prediction time during testing, and the sparse Bayesian inference allows for rapid parameter learning. Experimental results on several synthetic datasets, benchmark datasets, GPR-based landmine detection datasets, and human emotion recognition datasets show that our proposed approaches perform comparably or better than the existing approaches.</p> / Dissertation

No description available.

Ausgangspunkt der vorliegenden Arbeit ist die gut belegte Assoziation zwischen mütterlicher Depression und ungünstigen emotionalen und behavioralen Folgen für ihre Kinder. Allerdings sind die Faktoren, die zu der transgenerationalen Übertragung der Depression beitragen, noch nicht geklärt. Ziel dieser Arbeit ist es, zwei möglicherweise dazu beitragende psychologische Faktoren zu untersuchen: (1) Affekterkennung von Gesichtsausdrücken in Mutter-Kind Dyaden und (2) mütterliche emotionale Verfügbarkeit. Dazu wurden drei Studien durchgeführt. Studie I untersucht mittels funktioneller Magnetresonanztomographie (fMRT) unterscheidbare und überlappende Aktivierungsmuster bei gesunden Müttern, während sie fröhliche und traurige Gesichter ihres eigenen Kindes sehen. Studie II verwendet eine Morphing-Aufgabe, um die Affekterkennung in Müttern mit remittierter Depression und ihren Kindern zu untersuchen. In Studie III wird die emotionale Verfügbarkeit von Müttern mit remittierter Depression in einer Verhaltensbeobachtung untersucht. Ergebnisse der Studie I zeigen eine größere Gehirnaktivierung der Mütter bei traurigen eigenen Kindergesichtern in der Amygdala und anterioren Cingulum, hingegen bei fröhlichen im Hippocampus und inferioren Frontalgyrus. Überlappende Aktivierung wurde in der Insula gefunden. Diese Aktivierungsmuster könnten feinfühliges mütterliches Verhalten erleichtern und das Aufrechterhalten der Mutter-Kind Beziehung unterstützen. Ergebnisse von Studie II belegen einen negativen Verarbeitungsbias bei Müttern mit einer remittierten Depression, wobei parallele Veränderungen bei ihren Kindern gefunden wurden. Dies könnte auf einen transgenerationalen Übertragungsprozess hinweisen. Ergebnisse von Studie III zeigen eine verminderte emotionale Verfügbarkeit bei Müttern in Remission, was eine Trait-Eigenschaft darstellen könnte. / Starting point of the present dissertation is the well-established association between maternal depression and adverse emotional and behavioral outcomes in children. The factors contributing to the transgenerational transmission of depression have not been fully elucidated. The aim of this thesis is to investigate two psychological factors that potentially contribute to this transgenerational association: (1) affect recognition of facial expressions in mother-child dyads and (2) maternal emotional availability. Therefore, three studies have been conducted. In study I, functional magnetic resonance imaging (fMRI) is used to measure dissociable and overlapping brain activation in healthy mothers, while they view happy, neutral and sad faces of their own child. By using a morphing task, study II tests the hypothesis that affect recognition is biased in mothers with depression in remission and their children. Study III investigates whether emotional availability is reduced in mothers remitted from depression. Study I revealed greater brain activation in the amygdala and anterior cingulate cortex while mothers viewed sad faces of their own child, whereas greater brain activation was detected in the hippocampus and inferior frontal gyrus in response to happy faces. Conjoint activation was found in the insula. These activations might facilitate sensitive maternal behavior and promote mothers to maintain the mother-child relationship. Findings of study II demonstrate a negative processing bias in mothers with depression in remission, which was paralleled in their children. This finding could possibly point to a process of transgenerational transmission. Results of study III indicate reduced emotional availability in mothers who have remitted from depression, which might represent a trait characteristic of depression.

Les manifestations comportementales du sadisme commun et de la psychopathie sous-clinique suggèrent qu’ils seraient associés à des déficits affectifs en lien avec le traitement émotionnel. Il est proposé que les déficits affectifs associés au détachement émotionnel et au manque d’empathie chez les individus avec des tendances sadiques et les individus avec des traits psychopathiques seraient dus à des déficits au niveau de la reconnaissance émotionnelle faciale (REF). L’objectif principal de la présente étude était de clarifier l’association entre les tendances sadiques, les traits psychopathiques et la capacité de reconnaissance émotionnelle faciale. Le recrutement des participants s’est effectué dans des maisons de transition provinciales, dans un centre jeunesse, dans un centre d’intervention en délinquance sexuelle ainsi que dans un organisme venant en aide aux hommes aux prises avec des problématiques de violence. Les analyses statistiques ont été menées sur un échantillon se composant d’hommes délinquants (N = 81). La collecte des données s’est effectuée à l’aide de questionnaires en ligne comprenant des informations sociodémographiques ainsi que le Varieties of Sadistic Tendencies (VAST) et le Self-Report Psychopathy – Short Form (SRP-SF) ainsi que par une tâche de reconnaissance émotionnelle faciale en immersion virtuelle. Suite aux analyses descriptives, des analyses corrélationnelles de type r de Pearson ainsi que des analyses de régression linéaire multiple ont été menées sur l’ensemble des données. Les résultats suggèrent que les tendances sadiques n’étaient pas associées à des déficits au niveau de la reconnaissance émotionnelle faciale, au contraire, le score aux échelles du sadisme commun prédisait une meilleure habileté à reconnaître les expressions d’émotions. Les résultats suggèrent également que la composante de l’affect plat de la psychopathie serait associée avec une moins bonne performance pour la reconnaissance émotionnelle faciale des émotions en général ainsi qu’à un déficit spécifique au niveau de la reconnaissance de la tristesse. Les résultats sont interprétés en fonction des objectifs spécifiques de cette recherche. / The behavioral manifestations of everyday sadism and subclinical psychopathy suggest an association with emotional deficits and with emotional processing deficits. It is suggested that the emotional coldness and the lack of empathy associated with everyday sadism and subclinical psychopathy are due to facial affect recognition (FAR) deficits. The aim of this study is to assess the association between everyday sadism, subclinical psychopathy and facial affect recognition. The participants of the present study were recruited in provincial halfway houses, in youth centers, in an intervention center for sexual offenders and in an organization offering help to man with a history of violent behaviors. The statistical analyses were conducted on a sample comprised on male general offenders (N = 81). The collection of the data was made using an online questionnaire comprised of sociodemographic information, the Varieties of Sadistic Tendencies (VAST), the Self-Report Psychopathy – Short Form (SRP-SF) and a facial affect recognition task. Adding to descriptive analyses, Pearson’s r correlation analyses and multiple regression analyses were performed to assess the respective influence of sadistic tendencies and psychopathic traits on predicting facial affect recognition performance. No general nor specific facial affect recognition deficits were found in relation to sadism. On the opposite, sadism was associated with a better performance for overall facial affect recognition and sadness recognition. Results suggest that higher levels of callous affect are associated with a reduced overall facial affect recognition performance and a specific impairment for sadness recognition. The results are discussed according to the specific objectives of this study.

Mémoire de maîtrise présenté en vue de l'obtention de la maîtrise en psychologie (M. Sc) / Introduction. Il est bien établi que les survivants tumeurs cérébrales pédiatriques (STCP) éprouvent des difficultés sociales, et la reconnaissance d’émotions faciales a été étudiée comme un mécanisme sous-jacent. Cependant, l'influence possible de la pandémie sur les capacités de reconnaissance des affects chez les STCP reste inexplorée. La présente étude visait à comparer la reconnaissance des émotions faciales (avec accès au visage complet versus seulement la région des yeux) entre les STCP et des jeunes à développement typique ainsi qu’à examiner son association avec l'adaptation sociale. Méthode. Des STCP (n=23) au moins un an après le traitement et des contrôles (n=24) entre 8 et 16 ans ont complété le sous-test de reconnaissance des affects du NEPSY-II (visage complet) et la version enfant du Reading the Mind in the Eyes Test (RMET, seulement le haut du visage). Résultats. Les groupes ne différaient pas sur leurs habiletés de reconnaissance d’émotions et ceux-ci n’étaient pas associés à leur adaptation sociale. Comparé aux normes pré-pandémie, notre échantillon avait plus de difficultés dans leur capacité de reconnaissance d’émotions avec visage complet ainsi qu’une meilleure performance avec seulement le haut du visage disponible (p < .05). Les participants ont aussi obtenu de meilleurs résultats au RMET qu’au NEPSY-II (p< .05). Conclusion. En somme, la pandémie semble avoir joué un rôle sur les capacités de reconnaissance des émotions faciales, tant chez les STCP que chez les contrôles, soulignant la nécessité d'études futures sur les effets à long terme de la pandémie sur les compétences sociales des jeunes. / Introduction. It is well-established that pediatric brain tumour survivors (PBTS) experience social difficulties, and facial emotion recognition has been studied as an underlying mechanism. However, the possible influence of the pandemic on affect recognition abilities in PBTS remains unexplored. The present study aimed to compare facial affect recognition (with full versus partial facial features) between PBTS and healthy controls (HC) and to examine its association with social adjustment. Method. PBTS (N=23, ages 8-16) at least one-year post-treatment and HC (N=24, ages 8-16) completed the NEPSY-II Affect Recognition subtest (full face) and the child version of the Reading the Mind in the Eyes Test (RMET, upper face only). Results. The groups did not differ in their ability to recognize emotions, and these were not associated with social adjustment. Compared with pre-pandemic norms, our sample had a lower performance in their emotion recognition ability with full face and a better performance with only upper face (p < .05). Participants also performed better on the RMET than on the NEPSY-II (p< .05). Conclusion. In sum, the pandemic appears to have played a role in facial emotion recognition abilities in both PBTS and controls, highlighting the need for future studies on the pandemic long-term effects on young people's social skills.