No / How many times have you smiled today? How many times have you frowned today? Ever thought of being in a state of self-consciousness to be able to relate your own mood with your facial emotional expressions? Perhaps with our present-day busy lives, we may not consider these as crucial questions. However, as researchers uncover more and more about the human emotional landscape they are learning the importance of understanding our emotions.

Submitted by Automa??o e Estat?stica (sst@bczm.ufrn.br) on 2016-01-14T18:48:05Z No. of bitstreams: 1 GiulianaSilvaBezerra_DISSERT.pdf: 12899912 bytes, checksum: 413f2be6aef4a909500e6834e7b0ae63 (MD5) / Approved for entry into archive by Arlan Eloi Leite Silva (eloihistoriador@yahoo.com.br) on 2016-01-15T18:57:11Z (GMT) No. of bitstreams: 1 GiulianaSilvaBezerra_DISSERT.pdf: 12899912 bytes, checksum: 413f2be6aef4a909500e6834e7b0ae63 (MD5) / Made available in DSpace on 2016-01-15T18:57:11Z (GMT). No. of bitstreams: 1 GiulianaSilvaBezerra_DISSERT.pdf: 12899912 bytes, checksum: 413f2be6aef4a909500e6834e7b0ae63 (MD5) Previous issue date: 2014-12-12 / Emotion-based analysis has raised a lot of interest, particularly in areas such as forensics, medicine, music, psychology, and human-machine interface. Following this trend, the use of facial analysis (either automatic or human-based) is the most common subject to be investigated once this type of data can easily be collected and is well accepted in the literature as a metric for inference of emotional states. Despite this popularity, due to several constraints found in real world scenarios (e.g. lightning, complex backgrounds, facial hair and so on), automatically obtaining affective information from face accurately is a very challenging accomplishment. This work presents a framework which aims to analyse emotional experiences through naturally generated facial expressions. Our main contribution is a new 4-dimensional model to describe emotional experiences in terms of appraisal, facial expressions, mood, and subjective experiences. In addition, we present an experiment using a new protocol proposed to obtain spontaneous emotional reactions. The results have suggested that the initial emotional state described by the participants of the experiment was different from that described after the exposure to the eliciting stimulus, thus showing that the used stimuli were capable of inducing the expected emotional states in most individuals. Moreover, our results pointed out that spontaneous facial reactions to emotions are very different from those in prototypic expressions due to the lack of expressiveness in the latter. / Emotion-based analysis has raised a lot of interest, particularly in areas such as forensics, medicine, music, psychology, and human-machine interface. Following this trend, the use of facial analysis (either automatic or human-based) is the most common subject to be investigated once this type of data can easily be collected and is well accepted in the literature as a metric for inference of emotional states. Despite this popularity, due to several constraints found in real world scenarios (e.g. lightning, complex backgrounds, facial hair and so on), automatically obtaining affective information from face accurately is a very challenging accomplishment. This work presents a framework which aims to analyse emotional experiences through naturally generated facial expressions. Our main contribution is a new 4-dimensional model to describe emotional experiences in terms of appraisal, facial expressions, mood, and subjective experiences. In addition, we present an experiment using a new protocol proposed to obtain spontaneous emotional reactions. The results have suggested that the initial emotional state described by the participants of the experiment was different from that described after the exposure to the eliciting stimulus, thus showing that the used stimuli were capable of inducing the expected emotional states in most individuals. Moreover, our results pointed out that spontaneous facial reactions to emotions are very different from those in prototypic expressions due to the lack of expressiveness in the latter.

Sentiment analysis is a form of Natural Language Processing (NPL) withthe purpose of deducting and extracting sentiment and emotions fromwritten text. The interest in sentiment analysis emerged at the turn of themillennium and has increased steadily since 2004. This might be due to awish to take advantage of the enormous amounts of data generated on theinternet, and social media in particular, every single day.The purpose of the study has been to examine how applications usingsentiment analysis might be able to help autistic people navigate theemotional content in written communication. A qualitative study has beenconducted to understand what types of needs autistic people might haveon this type of application, how it might work and what it might looklike. Data has been collected through semistructured interviews. The datahas then been processed with the help of deductive thematic analysis.The result of the study is that the respondents, autistic people, imaginethat they might be helped by an application of this kind and the study hasfurthermore started exploring what type of functionality and designaspects might be desirable and less desirable. / Tonalitetsanalys är en form av Natural Language Processing (NPL) medsyfte att utläsa attityd och känslor ur skriven text. Intresset förtonalitetsanalys har funnits sedan millenieskiftet och ökat kraftigt sedan2004. Detta kan tänkas ha sin grund i en vilja att utnyttja de väldigadatamassorna som produceras på internet och kanske främst socialamedier varje dag.Syftet med studien har varit att undersöka hur applikationer som utförtonalitetsanalys på skriven kommunikation skulle kunna se ut för atthjälpa autistiska personer med att navigera känsloinnehåll. En studie medkvalitativ ansats har genomförts för att kunna förstå vilka behov och kravautistiska personer skulle ha på applikationer med detta syfte. Studiensempiriska data har samlats in genom semistrukturerade intervjuer medautistiska personer. Den insamlade datan har sedan behandlats med hjälpav deduktiv tematisk analysStudiens resultat visar att respondenterna, autistiska personer, anser sigkunna ha nytta av den här typen av applikationer och studien hardessutom börjat undersöka vilken typ av funktionalitet och designval somskulle vara önskvärda och icke önskvärda.

No description available.

We present results of a sentiment annotation study in the context of historical German plays. Our annotation corpus consists of 200 representative speeches from the German playwright Gotthold Ephraim Lessing. Six annotators, five non-experts and one expert in the domain, annotated the speeches according to different sentiment annotation schemes. They had to annotate the differentiated polarity (very negative, negative, neutral, mixed, positive, very positive), the binary polarity (positive/negative) and the occurrence of eight basic emotions. After the annotation, the participants completed a questionnaire about their experience of the annotation process; additional feedback was gathered in a closing interview. Analysis of the annotations shows that the agreement among annotators ranges from low to mediocre. The non-expert annotators perceive the task as very challenging and report different problems in understanding the language and the context. Although fewer problems occur for the expert annotator, we cannot find any differences in the agreement levels among non-experts and between the expert and the non-experts. At the end of the paper, we discuss the implications of this study and future research plans for this area

[pt] O interesse desta tese recai sobre compreender como os falantes de língua portuguesa a utilizam para materializar a menção de emoção através de um trabalho, sobretudo, linguístico. O objetivo geral da pesquisa é criar recursos para aprimorar a anotação do campo semântico das emoções na língua portuguesa a partir do projeto AC/DC, projeto que reúne e disponibiliza publicamente corpora anotados e recursos para pesquisas na língua portuguesa, e do Emocionário, projeto de anotação semântica e léxico de emoções. Inicialmente, a pesquisa dá um panorama dos estudos de emoção; se alinha às perspectivas que refutam a universalidade das emoções e abordagens que postulam emoções básicas; e contrapõe seu interesse por menção de emoção à já consolidada área de Análise de Sentimento, contrastando cinco léxicos de sentimento e/ou polaridades em língua portuguesa e o Emocionário. A partir de uma ampla varredura nos corpora do AC/DC, três principais caminhos foram percorridos para investigar palavras de emoção: (i) uma análise dos vinte e quatro grupos de emoção que já existiam no léxico do Emocionário a fim de delinear características e desafios no estudo de emoção na língua portuguesa; (ii) a revisão completa um terço dos grupos do léxico do Emocionário; e (iii) buscas pelo padrão léxico-sintático sentimento de N e por expressões anotadas pelo projeto Esqueleto usadas para descrever emoção. A análise dos corpora à luz dos lemas previamente pertencentes aos grupos do léxico do Emocionário evidenciou, dentre outras características, a relevância de expressões lexicalizadas para a análise da descrição de emoção, dos tipos de argumentos de verbos e afixos que podem causar variação de sentido, e de variações de tempo e modo verbal que acarretam mudança de significado. Dentre os desafios estão palavras e expressões polissêmicas e a dificuldade na detecção de diferentes sentidos em palavras que compartilham da mesma classe gramatical, tendo como base somente informações morfossintáticas. Esta análise possibilitou a estruturação e documentação de uma metodologia de revisão que pode vir a ser aplicada nos demais grupos futuramente. As principais contribuições desta tese são decorrentes das análises e explorações em corpora: a limpeza de lemas com sentidos não-emocionais dos grupos do léxico do Emocionário; a criação dos grupos de emoção Ausência e Outra, enriquecendo o léxico; a detecção de mais de novecentos lemas e expressões provenientes das buscas pelo padrão sentimento de N e das conexões estabelecidas entre os campos semânticos de emoção e do corpo humano; além de descobertas de campos lexicais pouco mencionados na literatura sobre emoção, como coletividade, estranhamento, espiritualidade, parentesco e atos automotivados, que auxiliaram na investigação de como os falantes do português cristalizam emoções na língua. / [en] The interest of this thesis lies in understanding how Portuguese speakers use it to materialize the mention of emotion through a linguistic perspective. The general objective of the research is to create resources to improve the annotation of the semantic field of emotions in the Portuguese language based on the AC/DC project, which gathers and makes publicly available annotated corpora and tools for linguistic research on Portuguese language. and Emocionário, which is both a semantic annotation project and lexicon of emotions. Initially, the research gives an overview of emotion studies; aligning itself with perspectives that refute the universality of emotions and approaches that postulate basic emotions; and contrasts the interest in emotion description to the already consolidated area of Sentiment Analysis, comparing five lexicons of emotion and/or polarities in Portuguese to Emocionário. From a broad sweep of the AC/DC corpora, three main paths were taken towards investigating emotion words: (i) an analysis of the twenty-four emotion groups previously composing the Emocionário lexicon in order to delineate characteristics and challenges in the study of emotion description in the Portuguese language; (ii) a thorough revision of one-third of the Emocionário lexicon groups; and (iii) searches for the lexical-syntactic pattern sentimento de N and for expressions annotated by the Esqueleto project used to describe emotion. The corpora analysis in the light of the lemmas previously belonging to the Emocionário lexicon groups showed, amongst other characteristics, the relevance of lexicalized expressions for the analysis of the emotion description, the types of arguments of verbs and affixes that can cause variation in meaning, and variations in tense and verbal mode that lead to a change in meaning. Amongst the challenges are polysemous words and expressions and the difficulty in detecting different meanings in words that share the same grammatical class, based only on morphosyntactic information. This analysis enabled the structuring and documentation of a revision methodology that may be applied in other groups in the future. The main contributions of this thesis derive from the analyzes and explorations in corpora: the exclusion of lemmas with non-emotional meanings from the Emocionário lexicon groups; the creation of emotion groups Ausência and Outra, enriching the lexicon; the detection of more than nine hundred lemmas and expressions from the searches for the sentimento de N pattern and the connections established between the semantic fields of emotion and the human body; in addition to discoveries of lexical fields rarely mentioned in the literature on emotion, such as coletividade, estranhamento, espiritualidade, parentesco e atos automotivados, which helped in the investigation of how Portuguese speakers crystallize emotions in language.

We present a case study as part of a work-in-progress project about multimodal sentiment analysis on historic German plays, taking Emilia Galotti by G. E. Lessing as our initial use case. We analyze the textual version and an audio version (audiobook). We focus on ready-to-use sentiment analysis methods: For the textual component, we implement a naive lexicon-based approach and another approach that enhances the lexicon by means of several NLP methods. For the audio analysis, we use the free version of the Vokaturi tool. We compare the results of all approaches and evaluate them against the annotations of a human expert, which serves as a gold standard. For our use case, we can show that audio and text sentiment analysis behave very differently: textual sentiment analysis tends to predict sentiment as rather negative and audio sentiment as rather positive. Compared to the gold standard, the textual sentiment analysis achieves accuracies of 56% while the accuracy for audio sentiment analysis is only 32%. We discuss possible reasons for these mediocre results and give an outlook on further steps we want to pursue in the context of multimodal sentiment analysis on historic plays.

Lors d'une intervention conversationnelle, le langage est supporté par une communication non-verbale qui joue un rôle central dans le comportement social humain en permettant de la rétroaction et en gérant la synchronisation, appuyant ainsi le contenu et la signification du discours. En effet, 55% du message est véhiculé par les expressions faciales, alors que seulement 7% est dû au message linguistique et 38% au paralangage. L'information concernant l'état émotionnel d'une personne est généralement inférée par les attributs faciaux. Cependant, on ne dispose pas vraiment d'instruments de mesure spécifiquement dédiés à ce type de comportements. En vision par ordinateur, on s'intéresse davantage au développement de systèmes d'analyse automatique des expressions faciales prototypiques pour les applications d'interaction homme-machine, d'analyse de vidéos de réunions, de sécurité, et même pour des applications cliniques. Dans la présente recherche, pour appréhender de tels indicateurs observables, nous essayons d'implanter un système capable de construire une source consistante et relativement exhaustive d'informations visuelles, lequel sera capable de distinguer sur un visage les traits et leurs déformations, permettant ainsi de reconnaître la présence ou absence d'une action faciale particulière. Une réflexion sur les techniques recensées nous a amené à explorer deux différentes approches. La première concerne l'aspect apparence dans lequel on se sert de l'orientation des gradients pour dégager une représentation dense des attributs faciaux. Hormis la représentation faciale, la principale difficulté d'un système, qui se veut être général, est la mise en œuvre d'un modèle générique indépendamment de l'identité de la personne, de la géométrie et de la taille des visages. La démarche qu'on propose repose sur l'élaboration d'un référentiel prototypique à partir d'un recalage par SIFT-flow dont on démontre, dans cette thèse, la supériorité par rapport à un alignement conventionnel utilisant la position des yeux. Dans une deuxième approche, on fait appel à un modèle géométrique à travers lequel les primitives faciales sont représentées par un filtrage de Gabor. Motivé par le fait que les expressions faciales sont non seulement ambigües et incohérentes d'une personne à une autre mais aussi dépendantes du contexte lui-même, à travers cette approche, on présente un système personnalisé de reconnaissance d'expressions faciales, dont la performance globale dépend directement de la performance du suivi d'un ensemble de points caractéristiques du visage. Ce suivi est effectué par une forme modifiée d'une technique d'estimation de disparité faisant intervenir la phase de Gabor. Dans cette thèse, on propose une redéfinition de la mesure de confiance et introduisons une procédure itérative et conditionnelle d'estimation du déplacement qui offrent un suivi plus robuste que les méthodes originales. / In a face-to-face talk, language is supported by nonverbal communication, which plays a central role in human social behavior by adding cues to the meaning of speech, providing feedback, and managing synchronization. Information about the emotional state of a person is usually carried out by facial attributes. In fact, 55% of a message is communicated by facial expressions whereas only 7% is due to linguistic language and 38% to paralanguage. However, there are currently no established instruments to measure such behavior. The computer vision community is therefore interested in the development of automated techniques for prototypic facial expression analysis, for human computer interaction applications, meeting video analysis, security and clinical applications. For gathering observable cues, we try to design, in this research, a framework that can build a relatively comprehensive source of visual information, which will be able to distinguish the facial deformations, thus allowing to point out the presence or absence of a particular facial action. A detailed review of identified techniques led us to explore two different approaches. The first approach involves appearance modeling, in which we use the gradient orientations to generate a dense representation of facial attributes. Besides the facial representation problem, the main difficulty of a system, which is intended to be general, is the implementation of a generic model independent of individual identity, face geometry and size. We therefore introduce a concept of prototypic referential mapping through a SIFT-flow registration that demonstrates, in this thesis, its superiority to the conventional eyes-based alignment. In a second approach, we use a geometric model through which the facial primitives are represented by Gabor filtering. Motivated by the fact that facial expressions are not only ambiguous and inconsistent across human but also dependent on the behavioral context; in this approach, we present a personalized facial expression recognition system whose overall performance is directly related to the localization performance of a set of facial fiducial points. These points are tracked through a sequence of video frames by a modification of a fast Gabor phase-based disparity estimation technique. In this thesis, we revisit the confidence measure, and introduce an iterative conditional procedure for displacement estimation that improves the robustness of the original methods.

Lors d'une intervention conversationnelle, le langage est supporté par une communication non-verbale qui joue un rôle central dans le comportement social humain en permettant de la rétroaction et en gérant la synchronisation, appuyant ainsi le contenu et la signification du discours. En effet, 55% du message est véhiculé par les expressions faciales, alors que seulement 7% est dû au message linguistique et 38% au paralangage. L'information concernant l'état émotionnel d'une personne est généralement inférée par les attributs faciaux. Cependant, on ne dispose pas vraiment d'instruments de mesure spécifiquement dédiés à ce type de comportements. En vision par ordinateur, on s'intéresse davantage au développement de systèmes d'analyse automatique des expressions faciales prototypiques pour les applications d'interaction homme-machine, d'analyse de vidéos de réunions, de sécurité, et même pour des applications cliniques. Dans la présente recherche, pour appréhender de tels indicateurs observables, nous essayons d'implanter un système capable de construire une source consistante et relativement exhaustive d'informations visuelles, lequel sera capable de distinguer sur un visage les traits et leurs déformations, permettant ainsi de reconnaître la présence ou absence d'une action faciale particulière. Une réflexion sur les techniques recensées nous a amené à explorer deux différentes approches. La première concerne l'aspect apparence dans lequel on se sert de l'orientation des gradients pour dégager une représentation dense des attributs faciaux. Hormis la représentation faciale, la principale difficulté d'un système, qui se veut être général, est la mise en œuvre d'un modèle générique indépendamment de l'identité de la personne, de la géométrie et de la taille des visages. La démarche qu'on propose repose sur l'élaboration d'un référentiel prototypique à partir d'un recalage par SIFT-flow dont on démontre, dans cette thèse, la supériorité par rapport à un alignement conventionnel utilisant la position des yeux. Dans une deuxième approche, on fait appel à un modèle géométrique à travers lequel les primitives faciales sont représentées par un filtrage de Gabor. Motivé par le fait que les expressions faciales sont non seulement ambigües et incohérentes d'une personne à une autre mais aussi dépendantes du contexte lui-même, à travers cette approche, on présente un système personnalisé de reconnaissance d'expressions faciales, dont la performance globale dépend directement de la performance du suivi d'un ensemble de points caractéristiques du visage. Ce suivi est effectué par une forme modifiée d'une technique d'estimation de disparité faisant intervenir la phase de Gabor. Dans cette thèse, on propose une redéfinition de la mesure de confiance et introduisons une procédure itérative et conditionnelle d'estimation du déplacement qui offrent un suivi plus robuste que les méthodes originales. / In a face-to-face talk, language is supported by nonverbal communication, which plays a central role in human social behavior by adding cues to the meaning of speech, providing feedback, and managing synchronization. Information about the emotional state of a person is usually carried out by facial attributes. In fact, 55% of a message is communicated by facial expressions whereas only 7% is due to linguistic language and 38% to paralanguage. However, there are currently no established instruments to measure such behavior. The computer vision community is therefore interested in the development of automated techniques for prototypic facial expression analysis, for human computer interaction applications, meeting video analysis, security and clinical applications. For gathering observable cues, we try to design, in this research, a framework that can build a relatively comprehensive source of visual information, which will be able to distinguish the facial deformations, thus allowing to point out the presence or absence of a particular facial action. A detailed review of identified techniques led us to explore two different approaches. The first approach involves appearance modeling, in which we use the gradient orientations to generate a dense representation of facial attributes. Besides the facial representation problem, the main difficulty of a system, which is intended to be general, is the implementation of a generic model independent of individual identity, face geometry and size. We therefore introduce a concept of prototypic referential mapping through a SIFT-flow registration that demonstrates, in this thesis, its superiority to the conventional eyes-based alignment. In a second approach, we use a geometric model through which the facial primitives are represented by Gabor filtering. Motivated by the fact that facial expressions are not only ambiguous and inconsistent across human but also dependent on the behavioral context; in this approach, we present a personalized facial expression recognition system whose overall performance is directly related to the localization performance of a set of facial fiducial points. These points are tracked through a sequence of video frames by a modification of a fast Gabor phase-based disparity estimation technique. In this thesis, we revisit the confidence measure, and introduce an iterative conditional procedure for displacement estimation that improves the robustness of the original methods.

Sentiment Analysis is an area of Computer Science that deals with the impact a document makes on a user. The very field is further sub-divided into Opinion Mining and Emotion Analysis, the latter of which is the basis for the present work. Work on songs is aimed at building affective interactive applications such as music recommendation engines. Using song lyrics, we are interested in both supervised and unsupervised analyses, each of which has its own pros and cons. For an unsupervised analysis (clustering), we use a standard probabilistic topic model called Latent Dirichlet Allocation (LDA). It mines topics from songs, which are nothing but probability distributions over the vocabulary of words. Some of the topics seem sentiment-based, motivating us to continue with this approach. We evaluate our clusters using a gold dataset collected from an apt website and get positive results. This approach would be useful in the absence of a supervisor dataset. In another part of our work, we argue the inescapable existence of supervision in terms of having to manually analyse the topics returned. Further, we have also used explicit supervision in terms of a training dataset for a classifier to learn sentiment specific classes. This analysis helps reduce dimensionality and improve classification accuracy. We get excellent dimensionality reduction using Support Vector Machines (SVM) for feature selection. For re-classification, we use the Naive Bayes Classifier (NBC) and SVM, both of which perform well. We also use Non-negative Matrix Factorization (NMF) for classification, but observe that the results coincide with those of NBC, with no exceptions. This drives us towards establishing a theoretical equivalence between the two.