Today's technology is becoming more interactive with voice assistants like Siri. However, interactive systems such as Siri make mistakes. The purpose of this thesis is to explore using affect as an implicit feedback channel so that such mistakes would be easily corrected in real time. The CAfFEINE Framework, which was created by Dr. Saha, is a context-aware affective feedback loop in an intelligent environment. For the research described in this thesis, the focus will be on analyzing a user's physiological response to the service provided by an intelligent environment. To test this feedback loop, an experiment was constructed using an on-screen, step-by-step assembly guide for a Tangram puzzle. To categorize the user's response to the experiment, baseline readings were gathered for a user's stressed and non-stressed state. The Paced Stroop Test and two other baseline tests were conducted to gather these two states. The data gathered in the baseline tests was then used to train a support vector machine to predict the user's response to the Tangram experiment. During the data analysis phase of the research, the results for the predictions on the Tangram experiment were not as expected. Multiple trials of training data for the support vector machine were explored, but the data gathered throughout this research was not enough to draw proper conclusions. More focus was then given to analyzing the pre-processed data of the baseline tests in an attempt to find a factor or group of factors to determine if the user's physiological responses would be useful to train the Support Vector Machine. There were trends found when comparing the area under the curves of the Paced Stroop Test phasic driver plots. It was found that these comparison factors might be a useful approach for differentiating users based upon their physiological responses during the Paced Stroop Test. / Master of Science / The purpose of this thesis was to use the CAfFEINE Framework, proposed by Dr. Saha, in a real-world environment. Dr. Saha's Framework utilizes a user's physical responses, i.e. heart rate, in a smart environment to give information to the smart devices. For example, if Siri were to give a user directions to someone's home and told that user to turn right when the user knew they needed to turn left. That user would have a physical reaction as in their heart rate would increase. If the user were wearing a smart watch, Siri would be able to see the heart rate increase and realize, from past experiences with that user, that the information she gave to the user was incorrect. Then she would be able to correct herself. My research focused on measuring user reaction to a smart service provided in a real-world situation using a Tangram puzzle as a mock version of an industrial assembly situation. The users were asked to follow on-screen instructions to assemble the Tangram puzzle. Their reactions were recorded through a smart watch and analyzed post-experiment. Based on the results of a Paced Stroop Test they took before the experiment, a computer algorithm would predict their stress levels for each service provided by the step-by-step instruction guide. However, the results did not turn out as expected. Therefore, the rest of the research focused more on why the results did not support Dr. Saha's previous Framework results.

Yes / Objective: Previous research has shown that the degree to which individuals base their intentions on particular underlying cognitions (i.e., motives) significantly moderates the intention-behavior relationship. Studies have individually examined the moderating effect of intentions based on overall attitude, affective attitudes, injunctive norms, and moral norms. The present research used a within-persons approach to simultaneously test the moderating effects of intentions based on instrumental attitude, affective attitude, anticipated affective reactions, injunctive norms, descriptive norms, and moral norms on the intention-behavior relationship and the impact of controlling for intention stability, self-efficacy and past behavior. Main Outcome Measures: Self-reported performance of 20 health behaviors. Design: Adults (N = 366) completed questionnaires assessing instrumental attitude, affective attitude, anticipated affective reactions, injunctive norms, descriptive norms, moral norms, self-efficacy and past behavior at baseline; intentions at baseline and one month follow-up; and behavior at two month follow-up for 20 health behaviors. Results: When tested simultaneously using multi-level modelling the only significant moderator of the intention-behavior relationship was the extent to which intentions were based on anticipated affective reactions (intentions more strongly based on anticipated affective reactions were significantly stronger predictors of behavior). This effect remained when we also controlled for intention stability (which also moderated the intention-behavior relationship), self-efficacy and past behavior. Conclusions: Intentions based on anticipated affective reactions may be particularly important predictors of health behaviors. Studies manipulating such intentions to test their effects on behavior change are required. / The research reported here was supported by a grant from the British Academy

<p>The Affective PDF Reader is a PDF Reader combined with affect recognition systems. The aim of the project is to research a way to provide the reader of a PDF with real - time visual feedback while reading the text to influence the reading experience in a positive way. The visual feedback is given in accordance to analyzed emotional states of the person reading the text - this is done by capturing and interpreting affective information with a facial expression recognition system. Further enhancements would also include analysis of voice in the computation as well as gaze tracking software to be able to use the point of gaze when rendering the visualizations.The idea of the Affective PDF Reader mainly arose in admitting that the way we read text on computers, mostly with frozen and dozed off faces, is somehow an unsatisfactory state or moreover a lonesome process and a poor communication. This work is also inspired by the significant progress and efforts in recognizing emotional states from video and audio signals and the new possibilities that arise from.The prototype system was providing visualizations of footprints in different shapes and colours which were controlled by captured facial expressions to enrich the textual content with affective information. The experience showed that visual feedback controlled by utterances of facial expressions can bring another dimension to the reading experience if the visual feedback is done in a frugal and non intrusive way and it showed that the evolvement of the users can be enhanced.</p>

This study drew upon the affective events and mood-congruent cognition theories to help explain how one domain influences the other. Affective events are things that happen to which people react emotionally and state affect is a result of those affective experiences. This study proposed that state affect generated in one domain would spillover and influence mood-congruent experiences in the receiving domain. Through an integration of organizational stressor-strain models (e.g., job-resources demand theory) and positive psychology, this study further proposed that positive events are resource-building and will work to prevent or buffer against strain responses to resource-depleting negative events. Finally, this study explored how individual differences in domain integration and work- and family-role salience moderate the foregoing relationships, particularly because studies investigating these effects have produced mixed results. To address these empirical questions, this study used the daily diary method to examine daily affective spillover effects from work-to-family and from family-to-work in a full-time working sample over the course of two weeks. This method was employed to help bolster confidence about the temporal precedence of work-family affective spillover and employee health and wellbeing outcomes. One-hundred and forty-four participants filled out diary questionnaires three times daily during the work week and one time daily during the weekend. Daily diaries assessed the participants’ exposure to a number of domain-specific affective events, state affect, physical symptoms, and sleep quality. Hierarchical linear modeling was used to test this study’s hypotheses. Overall, the results of this study support affective spillover as the linking pin between the two domains, which has health and wellbeing implications for employees. Specifically, tests of this study’s hypotheses indicated that exposure to affective events throughout the workday was related to state affect at the end of the workday, which then related to the number of valence-congruent affective events within the family domain. Exposure to those family-related affective events was related to corresponding changes in state affect, which not only persisted to the next morning but impacted employee health and wellbeing in terms of psychosomatic complaints. These findings are in line with both the affective events and mood-congruent theories. Only one significant moderating effect was observed. There was a positive relationship between negative affect at the end of the workday and the number of negative family affective events endorsed by participants who were lower on domain integration, but not among those who were higher on domain integration. The direction of this effect was surprising and may suggest that setting up strong boundaries between life domains creates unattainable expectations, which may increase negative outcomes for an employee. In sum, family-related affective experiences are an important variable to consider when investigating the effects of affective spillover on work-related experiences and health and wellbeing. The failure to do so may result in a considerable loss of information and contribute to mixed study results.

En France, l'éducation à la biodiversité est devenue une priorité au regard de la dégradation de la diversité biologique. À l'école primaire, elle est rattachée à la fois aux disciplines scientifiques et au développement durable. Les apprentissages attendus suivent essentiellement deux dimensions : les savoirs bio-écologiques et les comportements escomptés reliés à ces savoirs. Cependant, de nombreuses recherches font apparaître l'importance de la dimension affective dans ces apprentissages notamment au regard du rapport émotionnel entretenu avec les objets d'étude. Dans ce contexte, cette thèse explore les dimensions cognitive, affective et comportementale des apprentissages en éducation à la biodiversité. Le champ de recherche est restreint à l'étude d'arthropodes à l'école primaire. Comment la description et l'analyse de trois dimensions : savoir, affectivité et comportements permettent-elles de comprendre la construction des apprentissages en éducation à la biodiversité dans le cas de l'étude d'arthropodes à l'école primaire ? Pour répondre à cette question, le cadre conceptuel s'appuie sur l'analyse des savoirs, sur le modèle de la maturité affective vis-à-vis des objets étudiés et sur l'opérationalisation des comportements par les prises de position et intentions d'agir. Quatre objectifs sont poursuivis pour répondre à cette question : décrire et analyser chacune des trois dimensions, dégager les corrélations éventuelles entre les trois dimensions, identifier des profils particuliers d'apprenants, vérifier les effets d'une étude des arthropodes sur les apprentissages selon ces trois dimensions. La méthodologie utilisée suit deux perspectives : quantitative par une analyse multivariée des réponses à un questionnaire et qualitative par une analyse de réponses à la suite d'entretiens semi-dirigés. Les résultats obtenus ont permis de clarifier les trois dimensions considérées et d'éclairer leur articulation. La mesure et l'analyse de chaque dimension ont montré des différences significatives en fonction du type d'enseignement reçu en classe. Entre les dimensions analysées il existe des corrélations remarquables entre les savoirs, la maturité affective et les comportements. Des profils particuliers d'apprenants inter-reliant savoirs et affectivité ont pu être identifiés, ainsi que des profils généraux impliquant les trois dimensions. Nous avons pu constater l'importance de la relation affective avec les objets d'apprentissage qui, loin de perturber l'appropriation des savoirs aurait au contraire tendance à les renforcer tout en contribuant à construire des comportements citoyens et responsables. / In France, biodiversity education appears as a priority linked with the degradation of biodiversity. The biodiversity education in primary school is attached to both the scientific topics and sustainable development. The expected learning follows two basic dimensions: bio-ecological knowledges and behaviors related to the expected knowledges. However, numerous studies show the significance of affective dimension in learning, especially regarding the emotional relationship with the objects of study. In this context, this thesis explores the cognitive, affective and behavioral learning in biodiversity education. The search field is restricted to the study of arthropods in primary school. How the description and analysis of three dimensions: knowledge, emotions and behaviors allow to understand construction of learning in biodiversity education in the case of the study of arthropods in elementary school? To answer this question, conceptual framework is based on the analysis of knowledges, on the model of emotional maturity with the studied objects and the operationalization of behavior by positions and intentions to act. Four aims are proceeded to answer this issue: describe and analyze each of the three dimensions in the context of the study of arthropods in classroom, identify possible correlations between these three dimensions, identify specific patterns of learners and verify the effects of a study of arthropods on learning, according to these three dimensions. The methodology follows two perspectives: a quantitative multivariate analysis of survey responses and a qualitative analysis of answers during semi-structured interviews. The results of this study have clarified the three-dimensional knowledge, emotions and behaviors and highlighted the interplay between their relations. The measurement and analysis of each dimension pointed significant differences depending on the type of instruction received in classroom. There are significant correlations between scientific knowledge, emotional maturity and behavior according to the type of education. Specific profiles of learners linking knowledges and emotions have been identified, as well as general patterns involving the three dimensions. We noted the importance of emotional relationship with the learning objects that, instead of disrupting the knowledge, would rather tend to strengthen them while contributing to build citizenship and responsible behavior.

Les travaux de recherche de la thèse portent sur l'étude et la formalisation des interactions émotionnelles Humain-Machine. Au delà d’une détection d'informations paralinguistiques (émotions, disfluences,...) ponctuelles, il s'agit de fournir au système un profil interactionnel et émotionnel de l'utilisateur dynamique, enrichi pendant l’interaction. Ce profil permet d’adapter les stratégies de réponses de la machine au locuteur, et il peut également servir pour mieux gérer des relations à long terme. Le profil est fondé sur une représentation multi-niveau du traitement des indices émotionnels et interactionnels extraits à partir de l'audio via les outils de détection des émotions du LIMSI. Ainsi, des indices bas niveau (variations de la F0, d'énergie, etc.), fournissent des informations sur le type d'émotion exprimée, la force de l'émotion, le degré de loquacité, etc. Ces éléments à moyen niveau sont exploités dans le système afin de déterminer, au fil des interactions, le profil émotionnel et interactionnel de l'utilisateur. Ce profil est composé de six dimensions : optimisme, extraversion, stabilité émotionnelle, confiance en soi, affinité et domination (basé sur le modèle de personnalité OCEAN et les théories de l’interpersonal circumplex). Le comportement social du système est adapté en fonction de ce profil, de l'état de la tâche en cours, et du comportement courant du robot. Les règles de création et de mise à jour du profil émotionnel et interactionnel, ainsi que de sélection automatique du comportement du robot, ont été implémentées en logique floue à l'aide du moteur de décision développé par un partenaire du projet ROMEO. L’implémentation du système a été réalisée sur le robot NAO. Afin d’étudier les différents éléments de la boucle d’interaction émotionnelle entre l’utilisateur et le système, nous avons participé à la conception de plusieurs systèmes : système en Magicien d’Oz pré-scripté, système semi-automatisé, et système d’interaction émotionnelle autonome. Ces systèmes ont permis de recueillir des données en contrôlant plusieurs paramètres d’élicitation des émotions au sein d’une interaction ; nous présentons les résultats de ces expérimentations, et des protocoles d’évaluation de l’Interaction Humain-Robot via l’utilisation de systèmes à différents degrés d’autonomie. / Analysing and formalising the emotional aspect of the Human-Machine Interaction is the key to a successful relation. Beyond and isolated paralinguistic detection (emotion, disfluences…), our aim consists in providing the system with a dynamic emotional and interactional profile of the user, which can evolve throughout the interaction. This profile allows for an adaptation of the machine’s response strategy, and can deal with long term relationships. A multi-level processing of the emotional and interactional cues extracted from speech (LIMSI emotion detection tools) leads to the constitution of the profile. Low level cues ( F0, energy, etc.), are then interpreted in terms of expressed emotion, strength, or talkativeness of the speaker. These mid-level cues are processed in the system so as to determine, over the interaction sessions, the emotional and interactional profile of the user. The profile is made up of six dimensions: optimism, extroversion, emotional stability, self-confidence, affinity and dominance (based on the OCEAN personality model and the interpersonal circumplex theories). The information derived from this profile could allow for a measurement of the engagement of the speaker. The social behaviour of the system is adapted according to the profile, and the current task state and robot behaviour. Fuzzy logic rules drive the constitution of the profile and the automatic selection of the robotic behaviour. These determinist rules are implemented on a decision engine designed by a partner in the project ROMEO. We implemented the system on the humanoid robot NAO. The overriding issue dealt with in this thesis is the viable interpretation of the paralinguistic cues extracted from speech into a relevant emotional representation of the user. We deem it noteworthy to point out that multimodal cues could reinforce the profile’s robustness. So as to analyse the different parts of the emotional interaction loop between the user and the system, we collaborated in the design of several systems with different autonomy degrees: a pre-scripted Wizard-of-Oz system, a semi-automated system, and a fully autonomous system. Using these systems allowed us to collect emotional data in robotic interaction contexts, by controlling several emotion elicitation parameters. This thesis presents the results of these data collections, and offers an evaluation protocol for Human-Robot Interaction through systems with various degrees of autonomy.

A utilização de jogos sérios em muitas atividades, incluindo casos de saúde, como o processo de reabilitação motora, tem demonstrado resultados satisfatórios que encorajam o desenvolvimento de novas aplicações neste cenário. Jogos podem tornar tais atividades mais interessantes e divertidas para os pacientes, como também auxiliar as etapas do processo de reabilitação. Nestas aplicações, estratégias que visam a manutenção do nível de motivação do usuário durante a utilização são muito importantes. Assim, esta pesquisa investiga a adaptação de contexto em jogos sérios com a utilização de técnicas de Computação Afetiva. A proposta consiste em um framework que torna mais baixo ao programador o custo de implementação da adaptação afetiva em jogos e permite que o fisioterapeuta configure as adaptações que serão executadas no jogo conforme o perfil dos pacientes. Com o intuito de verificar a viabilidade da proposta, dois jogos para reabilitação motora e uma versão do framework foram implementados, permitindo a realização de experimentos com programadores, fisioterapeutas e pacientes. Os resultados obtidos permitem concluir que a abordagem proposta tende a proporcionar grande impacto social e tecnológico / The use of serious games in many activities, including health cases, like the motor rehabilitation process, has demonstrated results that encourage the development of new applications in this scenario. These activities can be more interesting and funnier by using games, as well as help the patients to execute the steps of the rehabilitation process. In these applications, strategies to maintain the user\'s motivation level during the game are very important. Thus, in this research, we investigated the context adaptation on serious games using techniques of Affective Computing. The proposal consists of a framework that makes the cost of implementing affective adaptation in games lower to programmers and allows the physiotherapists to configure the adaptations that will be executed in the game, according to the profile of the patients. In order to verify the feasibility of the proposal, two games for motor rehabilitation and a version of the framework were implemented, allowing the realization of experiments with programmers, physiotherapists, and patients. The results obtained allow us to conclude that the proposed approach tends to provide great social and technological impact

Uma das principais motivações da Inteligência Artificial no contexto dos sistemas de entretenimento digital é criar personagens adaptáveis a novas situações, pouco previsíveis, com aprendizado rápido, memória de situações passadas e uma grande diversidade de comportamentos consistente e convincente ao longo do tempo. De acordo com recentes estudos desenvolvidos nos campos da Neurociência e da Psicologia, a capacidade de resolução de problemas não está unicamente atrelada à facilidade na manipulação de símbolos, mas também à exploração das características do ambiente e à interação social, que pode ser expressa na forma de fenômenos emocionais. Os resultados desses estudos confirmam o papel fundamental que cumprem a personalidade e as emoções nas atividades de percepção, planejamento, raciocínio, criatividade, aprendizagem, memória e tomada de decisão. Quando módulos para a manipulação de personalidade e emoções são incorporados à teoria de agentes, é possível a construção de Agentes com Comportamento Convincente (Believable Agents). O objetivo principal deste trabalho é desenvolver e implementar uma arquitetura de agentes inteligentes para construir personagens sintéticos cujos estados afetivos influenciam em suas atividades cognitivas. Para o desenvolvimento de tal arquitetura utilizou-se o modelo BDI (Beliefs, Desires e Intentions) como base e aos módulos existentes em uma implementação desse modelo foi incluído um Módulo Afetivo. Esse Módulo Afetivo é constituído por três submódulos (Personalidade, Humor e Emoção) e deve impactar nas atividades cognitivas de percepção, memória e tomada de decisão do agente. Duas provas de conceito (experimentos) foram construídas : a simulação do problema do ``Dilema do Prisioneiro Iterado\'\' e a versão computadorizada do ``Jogo da Memória\'\'. A construção desses experimentos permitiu avaliar empiricamente a influência da personalidade, humor e emoção nas atividades cognitivas dos agentes, e consequentemente no seu comportamento. Os resultados evidenciam que a utilização da nova arquitetura permite a construção de agentes com comportamentos mais coerentes, adaptativos e cooperativos quando comparados aos de agentes construídos com arquiteturas cujas atividades cognitivas não consideram o estado afetivo, e também produz um comportamento mais próximo de um agente humano que de um comportamento ótimo ou aleatório. Essa evidência de sucesso, apresentada nos resultados, mostra que os agentes construídos com a arquitetura proposta nessa dissertação indicam um avanço na direção do desenvolvimento dos Agentes com Comportamento Convincente. / One of the main motivations of Artificial Intelligence in the context of the digital entertainment systems is to create characters that are adaptable to new situations, unpredictable, fast learners, enable with memory of past situations and a variety of consistent and convincing behavior over time. According to recent studies conducted in the fields of Neuroscience and Psychology, the ability to solve problems is not only related to the capacity to manipulate symbols, but also to the ability to explore the environment and to engage into social interaction, which can be expressed as emotional phenomena. The results of these studies confirm the key role the personality and emotions play in the activities of perception, attention, planning, reasoning, creativity, learning, memory and decision making. When modules for handling personality and emotion, are incorporated in a theory of agents, it is possible to build Believable Agents. The main objective of this work is to develop and implement an intelligent agent architecture to build synthetic characters whose affective states influence their cognitive activities. To develop such architecture the BDI model (Beliefs, Desires and Intentions) was used as a basis, to which an Affective Module was included. The Affective Module consists of three sub-modules (Personality, Mood and Emotion), which influence the cognitive activities of perception, memory and decision making. Finally, two proofs of concept were built: the simulation of the problem of ``Iterated Prisoner\'s Dilemma\'\' and the computerized version of the ``Memory Game.\'\' The construction of these experiments allowed to evaluate empirically the influence of personality, mood and emotion in cognitive activities of agents and consequently in their behavior. The results show that using the proposed architecture one can build agents with more consistent, adaptive and cooperative behaviors when compared to agents built with architectures whose affective states do not influence their cognitive activities. It also produces a behavior that is closer to a human user than that of optimal or random behavior. This evidence of success, presented in the obtained results, show that agents built with the proposed architecture indicate an advance towards the development of Believable Agents.

A dissertação “Cartografando memórias, vivências e sensibilidades – A Arte-Cidade-Universidade/UFRGS” propõe identificar como as práticas de arte e cultura promovidas pela UFRGS constroem relações com aqueles que experienciam a ação cultural. A pesquisa tem por fim materializar a intensidade da experiência do vivido nas ações do Projeto Unimúsica e como esse projeto foi capaz de marcar a memória das pessoas que com ele se relacionaram. Os rastros e as trilhas deste projeto foram buscados através de documentos, relatos, vídeos, imagens e sons. Procurou-se demonstrar que a permanência institucional de uma ação cultural, com uma proposta variada alcança uma intensidade passível de compor uma memória denominada “Cartografia afetiva” originada na relação que se estabeleceu entre indivíduos, a UFRGS e a cidade de Porto Alegre. / The dissertation “Cartographing memories, experiences and sensibilities – Art-City-University/UFRGS” proposes to identify how the practices of art and culture promoted by UFRGS build relationships with those who experience that cultural action. The research aims to materialize the intensity of the experience lived in the actions of the Project Unimúsica and how this project was able to mark the memory of the people who related to him. The traces and tracks of this project were searched through documents, reports, videos, images and sounds. It was tried to demonstrate that the institutional permanence of a cultural action, with a varied proposition reaches an intensity capable of composing a memory denominated “Affective cartography” originated in the relation that was established between individuals, the UFRGS and the city of Porto Alegre.

A utilização de jogos sérios em muitas atividades, incluindo casos de saúde, como o processo de reabilitação motora, tem demonstrado resultados satisfatórios que encorajam o desenvolvimento de novas aplicações neste cenário. Jogos podem tornar tais atividades mais interessantes e divertidas para os pacientes, como também auxiliar as etapas do processo de reabilitação. Nestas aplicações, estratégias que visam a manutenção do nível de motivação do usuário durante a utilização são muito importantes. Assim, esta pesquisa investiga a adaptação de contexto em jogos sérios com a utilização de técnicas de Computação Afetiva. A proposta consiste em um framework que torna mais baixo ao programador o custo de implementação da adaptação afetiva em jogos e permite que o fisioterapeuta configure as adaptações que serão executadas no jogo conforme o perfil dos pacientes. Com o intuito de verificar a viabilidade da proposta, dois jogos para reabilitação motora e uma versão do framework foram implementados, permitindo a realização de experimentos com programadores, fisioterapeutas e pacientes. Os resultados obtidos permitem concluir que a abordagem proposta tende a proporcionar grande impacto social e tecnológico / The use of serious games in many activities, including health cases, like the motor rehabilitation process, has demonstrated results that encourage the development of new applications in this scenario. These activities can be more interesting and funnier by using games, as well as help the patients to execute the steps of the rehabilitation process. In these applications, strategies to maintain the user\'s motivation level during the game are very important. Thus, in this research, we investigated the context adaptation on serious games using techniques of Affective Computing. The proposal consists of a framework that makes the cost of implementing affective adaptation in games lower to programmers and allows the physiotherapists to configure the adaptations that will be executed in the game, according to the profile of the patients. In order to verify the feasibility of the proposal, two games for motor rehabilitation and a version of the framework were implemented, allowing the realization of experiments with programmers, physiotherapists, and patients. The results obtained allow us to conclude that the proposed approach tends to provide great social and technological impact