Esta pesquisa é motivada pela crescente importância do reconhecimento automático de emoções, em especial através de sinais de voz, e suas aplicações em sistemas para interação homem-máquina. Neste contexto, são estudadas as emoções Felicidade, Medo, Nojo, Raiva, Tédio e Tristeza, além do estado Neutro, que são emoções geralmente consideradas como essenciais para um conjunto básico de emoções. São investigadas diversas questões relacionadas à análise de voz para reconhecimento de emoções, explorando vários parâmetros do sinal de voz, como por exemplo frequência fundamental (pitch), energia de curto prazo, formantes, coeficientes cepstrais e são testadas diferentes técnicas para a classificação, envolvendo reconhecimento de padrões e métodos estatísticos, como K-vizinhos mais próximos (KNN), Máquinas de Vetores de Suporte (SVM), Modelos de Misturas de Gaussianas (GMM) e Modelos Ocultos de Markov (HMM), destacando-se o uso de GMM como principal técnica utilizada por seu custo computacional e desempenho. Neste trabaho é desenvolvido um sistema de identificação em estágio único obtendo-se resultados superiores a diversos sistemas na literatura, com uma taxa de reconhecimento de até 74,86%. Além disso, recorre-se à psicologia e à teoria de emoções para incorporar-se a noção do espaço de emoções e suas dimensões a fim de desenvolver-se um sistema de classificação sequencial em três estágios, que passa por classificações nas dimensões Ativação, Avaliação e Domínio. Este sistema apresenta uma taxa de reconhecimento superior ao do sistema de único estágio, com até 82,41%, ao mesmo tempo em que é identificado um ponto de atenção no sistema de três estágios, que pode apresentar dificuldades na identificação de emoções que possuem baixo índice de reconhecimento em um dos estágios. Uma vez que existem poucos sistemas estado da arte que tratam o problema de verificação de emoções, um sistema também é desenvolvido para esta tarefa, obtendo-se um reconhecimento perfeito para as emoções Raiva, Neutro, Tédio e Tristeza. Por fim, é desenvolvido um sistema híbrido para tratar os problemas de verificação e de identificação em sequência, que tenta resolver o problema do classificador de três estágios e obtém uma taxa de reconhecimento de até 83%. / This work is motivated by the increase on the importance of automatic emotion recognition, especially through speech signals, and its applications in human-machine interaction systems. In this context, the emotions Happiness, Fear, Neutral, Disgust, Anger, Boredom and Sadness are selected for this study, which are usually considered essential for a basic set of emotions. Several topics related to emotion recognition through speech are investigated, including speech features, like pitch, energy, formants and MFCC as well as different classification algorithms that involve pattern recognition and stochastic modelling like K-Nearest Neighbours (KNN), Support Vector Machines (SVM), Gaussian Mixture Models (GMM) and Hidden Markov Models (HMM), where GMM is selected as the main technique for its computational cost and performance. In this work, a single-stage identification system is developed, which outperforms several systems in the literature, with a recognition rate of up to 74.86%. Besides, the idea of emotional space dimensions from Psychology and Emotion Theory is reviewed for the development of a sequential classification system with 3 stages, that passes through classifications on the Activation, Evaluation and Dominance dimensions. This system outperforms the single-stage classifier with a recognition rate of up to 82.41%, at the same time as a point of attention is identified, as this kind of system may show difficulties on the identification of emotions that show low recognition rates in a specific stage. Since there are few state of the art systems that handle emotion verification, a system for this task is also developed in this work, showing itself to be a perfect recognizer for the Anger, Neutral, Boredom and Sadness emotions. Finally, a hybrid system is proposed to handle both the verification and the identification tasks sequentially, which tries to solve the 3-stage classifier problem and shows a recognition rate of up to 83%.

Esta pesquisa é motivada pela crescente importância do reconhecimento automático de emoções, em especial através de sinais de voz, e suas aplicações em sistemas para interação homem-máquina. Neste contexto, são estudadas as emoções Felicidade, Medo, Nojo, Raiva, Tédio e Tristeza, além do estado Neutro, que são emoções geralmente consideradas como essenciais para um conjunto básico de emoções. São investigadas diversas questões relacionadas à análise de voz para reconhecimento de emoções, explorando vários parâmetros do sinal de voz, como por exemplo frequência fundamental (pitch), energia de curto prazo, formantes, coeficientes cepstrais e são testadas diferentes técnicas para a classificação, envolvendo reconhecimento de padrões e métodos estatísticos, como K-vizinhos mais próximos (KNN), Máquinas de Vetores de Suporte (SVM), Modelos de Misturas de Gaussianas (GMM) e Modelos Ocultos de Markov (HMM), destacando-se o uso de GMM como principal técnica utilizada por seu custo computacional e desempenho. Neste trabaho é desenvolvido um sistema de identificação em estágio único obtendo-se resultados superiores a diversos sistemas na literatura, com uma taxa de reconhecimento de até 74,86%. Além disso, recorre-se à psicologia e à teoria de emoções para incorporar-se a noção do espaço de emoções e suas dimensões a fim de desenvolver-se um sistema de classificação sequencial em três estágios, que passa por classificações nas dimensões Ativação, Avaliação e Domínio. Este sistema apresenta uma taxa de reconhecimento superior ao do sistema de único estágio, com até 82,41%, ao mesmo tempo em que é identificado um ponto de atenção no sistema de três estágios, que pode apresentar dificuldades na identificação de emoções que possuem baixo índice de reconhecimento em um dos estágios. Uma vez que existem poucos sistemas estado da arte que tratam o problema de verificação de emoções, um sistema também é desenvolvido para esta tarefa, obtendo-se um reconhecimento perfeito para as emoções Raiva, Neutro, Tédio e Tristeza. Por fim, é desenvolvido um sistema híbrido para tratar os problemas de verificação e de identificação em sequência, que tenta resolver o problema do classificador de três estágios e obtém uma taxa de reconhecimento de até 83%. / This work is motivated by the increase on the importance of automatic emotion recognition, especially through speech signals, and its applications in human-machine interaction systems. In this context, the emotions Happiness, Fear, Neutral, Disgust, Anger, Boredom and Sadness are selected for this study, which are usually considered essential for a basic set of emotions. Several topics related to emotion recognition through speech are investigated, including speech features, like pitch, energy, formants and MFCC as well as different classification algorithms that involve pattern recognition and stochastic modelling like K-Nearest Neighbours (KNN), Support Vector Machines (SVM), Gaussian Mixture Models (GMM) and Hidden Markov Models (HMM), where GMM is selected as the main technique for its computational cost and performance. In this work, a single-stage identification system is developed, which outperforms several systems in the literature, with a recognition rate of up to 74.86%. Besides, the idea of emotional space dimensions from Psychology and Emotion Theory is reviewed for the development of a sequential classification system with 3 stages, that passes through classifications on the Activation, Evaluation and Dominance dimensions. This system outperforms the single-stage classifier with a recognition rate of up to 82.41%, at the same time as a point of attention is identified, as this kind of system may show difficulties on the identification of emotions that show low recognition rates in a specific stage. Since there are few state of the art systems that handle emotion verification, a system for this task is also developed in this work, showing itself to be a perfect recognizer for the Anger, Neutral, Boredom and Sadness emotions. Finally, a hybrid system is proposed to handle both the verification and the identification tasks sequentially, which tries to solve the 3-stage classifier problem and shows a recognition rate of up to 83%.

L’objectif de cette recherche est d’étudier les processus d’activation et d’inhibition lexico-émotionnelles dans le vieillissement. Plusieurs études suggèrent un déficit des processus d’activation et d’inhibition lexicales lors de l’avancée en âge. Cependant, de nombreux travaux évoquent l’hypothèse d’une meilleure régulation émotionnelle dans le vieillissement visant à maintenir un bien-être subjectif constant. Ce maintien du bien-être subjectif entrainerait chez les adultes âgés, une sensibilité accrue envers les stimuli positifs tandis que les stimuli négatifs seraient désengagés. Dans cette thèse, il s’agit de préciser les processus d’activation et d’inhibition lexico-émotionnelles en présence d’un contexte fortement prédictible lors de tâches impliquant la complétion de phrases (Expérience 1, 2a) ou la reconnaissance des mots (Expérience 3a). Globalement, les résultats suggèrent une atteinte conjointe des processus d’activation et d’inhibition lexicales chez les adultes âgés comparés aux adultes jeunes, ainsi qu’un traitement privilégié de la valence positive des mots, tandis que les mots négatifs semblent désengagés. Dans un second temps, nous avons testés l’hypothèse d’une mobilisation de l’attention spécifique en fonction de l’âge et de la valence des mots à un niveau automatique (Expérience 4, 5a) ainsi que lors de la mobilisation attentionnelle volontaire (Expérience 6). Les résultats suggèrent une mobilisation attentionnelle à un niveau automatique mais aussi contrôlé envers les stimuli positifs dans le vieillissement normal. Dans l’ensemble, les données obtenues sont compatibles avec la littérature suggérant chez les adultes âgés une diminution de l’efficience des processus d’activation et d’inhibition lexicales, ainsi qu’une régulation émotionnelle effective qui s’étend au domaine du langage. La compréhension des processus lexico-émotionnels dans le vieillissement normal pourrait être étendue aux pathologies du vieillissement telle la maladie de Parkinson (Expériences 2b, 3b et 5b) afin de mieux comprendre le rôle et l’évolution du système affectif dans ces pathologies. / The aim of this research is to study lexico-emotional processes of activation and inhibition in aging. Studies suggest that normal aging is characterized by lexical activation and inhibition impairment. Nonetheless, recent findings showed that emotional regulation tend to enhance with aging in order to maintain subjective well-being effective. This subjective well-being could enhance sensitivity in older adults toward positive stimuli whereas negative ones could be disengaged. We investigate lexico-emotional activation and inhibition processing with high predictable sentence context in a sentence completion task (Experiment 1, 2a) and in word recognition (Experiment 3a). As a whole, results indicate in older adults both activation and inhibition impairment, and also better performances toward positive stimuli whereas negative ones seem to be disengaged compared to younger ones. Furthermore, we hypothesized an automatic attention allocation (Experiment 4, 5a) and a voluntary attention allocation (Experiment 6) toward positive stimuli than negative. Results obtained reported an attentional bias toward positive stimuli in aging. Generally, it seems that our results are in line with literature concerning both-damage of lexical activation and inhibition processes, and also in the enhancement of positive stimuli treatment in aging. Understanding of lexico-emotional processing in normal aging could be extended to pathology like Parkinson’s disease (Experiments 2b, 3b and 5b) in order to understand affective system role’s in pathological aging.

Children with language impairment (LI) have demonstrated several language problems, including receptive and expressive deficits. A growing body of research has further demonstrated emotion understanding and, more specifically, emotion inferencing deficits that negatively affect the ability of this population to use language successfully in social interactions. Consequently, the present study examines social communication intervention focused on improving emotion inferencing for children with a diagnosis of LI or Autism Spectrum Disorder (ASD), as similar social communication deficits are occasionally seen in children with these diagnoses. Study participants were identified from the caseload of a practicing certified speech-language pathologist (SLP) and qualified for intervention based upon Comprehensive Assessment of Spoken Language (CASL) and Universal Nonverbal Intelligence Test (UNIT) scores. Participant ability to perform an emotion inferencing task was then compared pre- and post-treatment with descriptive statistics and presented as 6 individual case studies to determine the effectiveness of social communication intervention. Results are presented pre- and post-intervention and indicated that emotion inferencing capacities for a given scenario increased for some participants, while others demonstrated either modest gains or slight declines in performance after intervention.

Étant donné l’investissement économique que représente le développement de produits innovants ainsi que leur faible taux de succès auprès du grand public, anticiper ce succès est devenu essentiel pour les industriels. Par conséquent, un cadre d’analyse centré sur l’expérience utilisateur met en avant l’émotion des individus comme critère déterminant pour prédire l’utilisation future d’un produit. Néanmoins, malgré les avantages des modèles issus de ce cadre d’analyse, ils apparaissent également comme étant limités du point de vue des prédictions qu’ils permettent de réaliser. Parallèlement, d’autres modèles issus de la théorie des intentions ont cherché à identifier les prédicteurs de l’intention comportementale des individus à utiliser pour la première fois ces produits ou bien ceux de l’intention comportementale des individus à les utiliser de nouveau. Ces modèles, comme le Technology Acceptance Model, l’Unified Theory of Acceptance and Use of Technology et le Consumer Acceptance of Technology, mettent en lien les attitudes envers les caractéristiques des produits et ces intentions d’utilisation. Cependant, ils posent également quelques problèmes vis-à-vis de leurs manières de prendre en compte l’émotion des individus.Pour répondre à la problématique de l’étude de l’émotion suscitée par des produits, nous proposons d’adopter le Component Process Model. Selon ce modèle l’émotion est le résultat de modifications internes et externes qui peuvent être regroupées en cinq composantes : la composante cognitive, la composante motivationnelle, la composante subjective, la composante expressive et la composante physiologique. Nous proposons d’utiliser ce modèle pour étudier spécifiquement l’influence de l’innovation des produits sur les émotions. Ainsi dans nos Études 1 et 2, nous monterons d’une part qu’il est possible d’analyser l’émotion suscitée par des produits avant même leur utilisation d’après trois des cinq composantes et d’autre part que ces émotions ne sont pas influencées par des caractéristiques propres aux individus. Ensuite dans nos Études 3 et 4, nous testerons expérimentalement ces composantes en comparant les émotions suscitées par les toutes premières utilisations de produits dits « innovants » avec l’utilisation de produits dits « communs ». Les résultats de ces études confirmeront l’intérêt qu’a l’utilisation du Component Process Model dans l’étude des émotions suscitées par des produits et plus particulièrement par des produits innovants. / Given the sizable economic investment of innovative products’ development, anticipating their success has become essential for companies. Therefore, a descriptive framework centered on user experience – also called UX – highlights individuals’ emotions as a criterion to predict the future use of products. Nevertheless, despite the pros of the models derived from this framework, they also have limits regarding the predictions they allow to make. Meanwhile, other psychological models aim to identify predictors of individuals’ behavioral intention to use these products for the first time or to use them again. These models – such as the Technology Acceptance Model, the Unified Theory of Acceptance and Use of Technologies or the Consumer Acceptance of Technology – enable to foresee individuals’ attitudes towards product characteristics and their intention to use them. However, they are also problematic regarding their way to take users’ emotions into account.To address the problem of the study of emotions triggered by products, we propose to adapt the Component Process Model. In this model emotions are the result of internal and external changes that can be categorized into five components: the cognitive component, the motivational component, the subjective component, the expressive component and the physiological component. We suggest using this model to specifically study the influence of a product’s innovational impact on emotions. Thus, in Studies 1 and 2 we show; firstly that it is possible to analyze the emotion elicited by products before their uses from three of the five components and secondly, that these emotions are not influenced by individuals’ personality and context. Then, in Studies 3 and 4, we experimentally test these components by comparing the emotions triggered by the first uses of so-called “innovative” products compared to the uses of products known as being “traditional”. The results of these studies confirm the advantage of using the Component Process Model to analyze emotions triggered by products and, more particularly, by innovative products.

Adolescents engage in risk behaviors at an alarming rate, and particularly when they are with peers. Despite efforts to develop prevention and intervention programs, rates of risk-taking among adolescents is still relatively high. Adolescents continue to engage in physical fights and aggressive behavior, use substances such as alcohol and illicit drugs, and make decisions that impact driving and motor vehicle incidents. The regulation of emotions plays a significant role in adolescents’ decisions to engage in such risk behaviors.Examining adolescents’ emotion expression and regulation is therefore critical to identifying ways to support positive development. This dissertation project explored important regulatory mechanisms that underlie adolescents’ behavior in 108 adolescents, by examining synchrony between emotion expression and physiological arousal (change from baseline heart rate to tasking heart rate)during a risk-taking task. The study also assessed the extent to which the social context of peers shifts emotion expression and physiological arousal, and whether this is associated with adolescents’ behavioral and social outcomes, and if these associations vary by gender. Study results suggest that the presence of a peer influences adolescents’ emotion expression. Specifically, adolescents showed greater expression of emotion when completing a risk-taking task in the presence of a peer, than when they completed the task alone. Additionally, adolescent girls are generally more expressive than their male counterparts and equally expressive alone and with a peer, but adolescent boys express more when they are with a peer than when they are alone. Synchrony between emotion expression and physiological arousal was not evident, however results of supplemental analyses suggest that physiological arousal (change from baseline heart rate to heart rate during the task)plays a moderating role in the association between emotion expression and social competence. Findings from the proposed study may inform intervention and policy efforts to understand and promote positive development among adolescents. In particular, results may shift how adults understand and respond to adolescent behavior in social contexts such as classrooms.

Music is a powerful and reliable means to stimulate the percept of both intense pleasantness and unpleasantness in the perceiver. However, everyone’s social experiences with music suggest that the same music piece may elicit a very different valence percept in different individuals. A comparison of music from different historical periods suggests that enculturation modulates the valence percept of intervals and harmonies, and thus possibly also of relatively basic feature extraction processes. Strikingly, it is still largely unknown how much the valence percept is dependent on physical properties of the stimulus and thus mediated by a universal perceptual mechanism, and how much it is dependent on cultural imprinting. The current thesis investigates the neurophysiology of the valence percept, and the modulating influence of culture on several distinguishable sub-processes of music processing, so-called functional modules of music processing, engaged in the mediation of the valence percept. / Musik eignet sich besonders gut, um sowohl intensive Angenehmheit/Lust und Unangenehmheit/Unlust (siehe auch Wundt, 1896), so genannte Valenzperzepte, im Zuhörer hervorzurufen. Jedoch kann derselbe musikalische Stimulus sehr unterschiedliche Valenzperzepte in verschiedenen Zuhörern hervorrufen, was nahe legt, dass das durch Musik vermittelte Valenzperzept zumindest teilweise durch kulturelle Prägung moduliert wird. Ein Vergleich von Musik verschiedener historischer Perioden legt ebenfalls nahe, dass kulturelle Prägung das Valenzperzept des Hörers bei der Wahrnehmung von Intervallen und Harmonien moduliert. Wichtigerweise ist es nach wie vor weitgehend unbekannt, inwiefern das Valenzperzept von physikalischen Eigenschaften des Stimulus (z.B. Rauhigkeit) abhängt - und daher auf einem universellen perzeptiven Mechanismus basiert - oder wie sehr es abhängt von kultureller Prägung. Die vorliegende Dissertation untersucht die Neurophysiologie des Valenzperzepts, sowie den modulierenden Einfluss von Kultur auf mehrere funktionelle Module der Musikwahrnehmung (voneinander unterscheidbare Subprozesse der Musikwahrnehmung), die bei der Entstehung des Valenzperzepts beteiligt sind.

In dieser konversationsanalytisch-interaktionslinguistischen Arbeit werden verbale, para-verbale und nonverbale Ressourcen zur Darstellung des Affekts Mitleid im Rahmen der sozialen Aktivität Erzählung untersucht. Grundlage der Analyse sind Video- und Audiodaten deutscher Gespräche. Es wird einerseits aufgezeigt, welche Ressourcen von Erzählerinnen eingesetzt werden, um affektive Reaktionen relevant zu machen (z.B. Blickverhalten, Augenbrauenbewegungen, rhetorische Mittel), andererseits wird beschrieben, welche spezifischen Ressourcen zur Darstellung von Mitleid eingesetzt werden (z.B. Interjektionen mit spezifischen Tonhöhenverläufen, „arme/r/s + S“-Konstruktionen, Handbewegungen). Es wird zudem demonstriert, dass es sich bei der Darstellung von Mitleid um ein interaktiv hergestelltes, soziales Phänomen handelt. Anhand des metakommunikativen Wissens der Sprecher, das sprachlich relevant gemacht wird und somit für die Analyse zugänglich ist, kann eine grundlegende soziale Regel zur Darstellung von Mitleid herausgearbeitet werden: Voraussetzung für Mitleid bzw. seine Darstellung ist eine spezifische Qualität der sozialen Beziehung zu der ‚consequential figure‘ (Maynard 1997). Nur wenn diese vorhanden ist bzw. interaktiv hergestellt wird, kann eine Mitleidsdarstellung in einer Erzählung über problematische Themen relevant gemacht werden und der lokalen Herstellung von sozialer Nähe zwischen den Gesprächsteilnehmern dienen. / The thesis explores the verbal, para-verbal and nonverbal resources which speakers deploy in everyday conversation to display sympathy in the course of the social activity storytelling. The analysis draws upon Conversation Analysis and Interactional Linguistics and is based on video and audio data of German talk-in-interaction. It will be shown which resources can be used by storytellers to make affective reactions relevant (e.g. gaze, eye brow movements, rhetorical devices) and which resources are deployed by recipients to display sympathy (e.g. interjections with specific pitch contours, ‘poor+N’-constructions, hand gestures). It will also be demonstrated how participants manage the transition from problematic phases of storytelling to subsequent talk, e.g. by contextualizing it as more humorous. Furthermore, participants provide access to their meta-communicative knowledge of when and how to display sympathy appropriately by making it linguistically relevant in the interaction. Thus, from these observations, it is possible to assume certain feeling and/or display rules for this specific kind of emotive involvement. The overall findings of this study are that a) it seems that a specific quality of the relationship to the ‘consequential figure’ (Maynard 1997) is a prerequisite for displaying sympathy, and b) social closeness may be created locally within the interaction through the affect display.

This study analyzed racial differences in the way African Americans and Caucasians perceive emotion from facial expressions and tone of voice. Participants were African American (n=25) and Caucasian (n=26) college students. The study utilizes 56 images of African American and Caucasian faces balanced for race and sex from the NimStim stimulus set (Tottenham, 2006). The study also utilized visual and auditory stimuli form the DANVA2. Participants were asked to judged emotion for each stimulus in the tasks. The BFRT, the WASI, and the Seashore Rhythm test were used as exclusionary criteria. In general the study found few differences in the way African Americans and Caucasians perceived emotion, though racial differences emerged as an interaction with other factors. The results of the study supported the theory of universality of emotion perception and expression though social influences, which may affect emotion perception, is also a possibility. Areas of future research were discussed.

This study estimated the association between aggressive behavior and two different types of emotion regulation, one operating on the conscious level with voluntary effort (i.e., suppression) and the other operating on the unconscious level with involuntary effort, or automatically (i.e., repression). Results from a correlation analysis among self-assessed suppression and repression and other-rated aggressive behavior showed that repression is more significantly linked to aggressive behavior than suppression. Further investigation using physiological and neural assessments was performed to determine the critical properties, including cardiac reactivity and neural substrates, of repression related to aggressive behavior. Based on the findings from multiple approaches in assessment, this study suggests that unconscious emotion change inferred from self-assessed repression (in Study 1) and neural activity (in Study 2) more significantly predicts aggressive behavior than personality. Implications for both aggression and emotion research are discussed along with the measurement equivalence issue.