Touch is an expressive and powerful modality in affect conveyance. A simple touch like a hug can elicit strong feelings of affection both in the touch initiator and recipient. Therefore delivering touch over a distance to a long-distance family member or significant other has been an appealing concept for both researchers and designers. However compared to the development of audio, video channels which allow the transmission of voice, facial expression and gesture, digitally mediated touch (Remote Touch) has not received much attention. We believe that this is partially due to the lack of understanding of the capabilities and communication possibilities that remote touch brings. This dissertation presents a review of relevant psychological and sociological literature of touch and proposes a model of immediacy of the touch channel for affect conveyance. We advance three hypotheses regarding the possibility of remote touch in immediate affect conveyance: presence, fidelity and context. We posit that remote touch with relatively low touch fidelity can convey meaningful immediate affect when it is accompanied by a contextualizing channel. To test the hypothesis, two sets of remote touch devices are designed and prototyped which allow users to send/receive a squeeze on the upper arm to/from others effectively. Three in-lab user studies are conducted to investigate the role of remote touch in affect conveyance. These studies showed clearly that remote touch, when contextualized, can influence the affective component in communication. Our results demonstrated that remote touch can afford a rich spectrum of meanings and affects. Three major categories of the usage are identified as positive affect touch which serves to convey affects such as affection, sympathy and sharing, comfort etc., playful touch which serves to lighten the conversations, and conversational touch which serves to regulate the dynamics in the discourse. Our interview results also provide insights of how people use this new channel in their communication. / Ph. D.

This field study examined how elderly and cognitively disabled people used and experienced a social companion robot. The following pages explores the questions: Which are the physical, social and affective qualities during the interaction? The aim was to through observations see how qualities of interaction could activate different forms of behavior. The results show that motion, sound and the eyes together created communicative and emotional changes for users who felt joy and were willing to share the activity with others. The robot stimulated to some extent users to create their own imaginative experiences but often failed to involve user or group for a long time and was also considered too large and heavy to handle. / Denna fältstudie undersökte hur äldre och kognitivt funktionsnedsatta personer använde och upplevde en social robot. Följande sidor utforskar frågorna: Vilka fysiska, sociala och affektiva kvaliteter finns i interaktionen? Målet var att genom observationer se hur kvaliteterna i interaktionen kunde aktivera olika typer av beteenden. Resultatet visar att rörelse, ljud och ögon tillsammans skapade kommunikativa och känslomässiga förändringar hos användarna som visade glädje och som gärna delade upplevelsen med andra. Roboten stimulerade till viss del användarna att skapa egna fantasifulla upplevelser men lyckades inte ofta involvera användare eller grupp under någon längre tid och ansågs även vara för stor och tung att hantera.

This dissertation work considers communication between people. I look at coordinating dyads (couples in relationships) and people in working relationships to develop an understanding of how people engage in high-stakes, or emotional communication via various communicative media. The approach for this research is to observe and measure people's behavior during interaction and subsequent reporting of that behavior and associated internal experiences. Qualitative and quantitative methods are employed. Quantitative data are analyzed using a range of statistical analyses, including correlations matrices, ANOVAs, and multivariate statistics. Two controlled laboratory experiments were conducted for this research. These experiments involved couples in relationships. Couples were brought into the lab and argued with each other across one of three technological media: face-to-face, telephone, and instant messaging (IM). In one set of couples' experiments, the couples argued for twenty minutes; in the subsequent couples' experiment, couples were encouraged to take as much time as they needed for their arguments. One of the main results from the first experiment is that couples did, indeed, argue when brought into a laboratory setting. One of the important findings for the second experiment is that time did not affect couples' tendency to reach closure during their arguments. This research is a contribution in that it examines how people engage in highly emotional communication using various technological media. In a society with ever-increasing communication needs that require technology, it becomes necessary to study its communicative affordances. Understanding the context of highly emotional interactions between members of couples gives insight into how technology meets (or fails to meet) these communication needs. / Ph. D.

Les émotions et leurs expressions par des agents virtuels sont deux enjeux importants pour les interfaces homme-machine affectives à venir. En effet, les évolutions récentes des travaux en psychologie des émotions, ainsi que la progression des techniques de l’informatique graphique, permettent aujourd’hui d’animer des personnages virtuels réalistes et capables d’exprimer leurs émotions via de nombreuses modalités. Si plusieurs systèmes d’agents virtuels existent, ils restent encore limités par la diversité des modèles d’émotions utilisés, par leur niveau de réalisme, et par leurs capacités d’interaction temps réel. Dans nos recherches, nous nous intéressons aux agents virtuels capables d’exprimer des émotions via leurs expressions faciales en situation d’interaction avec l’utilisateur. Nos travaux posent de nombreuses questions scientifiques et ouvrent sur les problématiques suivantes : Comment modéliser les émotions en informatique en se basant sur les différentes approches des émotions en psychologie ? Quel niveau de réalisme visuel de l’agent est nécessaire pour permettre une bonne expressivité émotionnelle ? Comment permettre l’interaction temps réel avec un agent virtuel ? Comment évaluer l’impact des émotions exprimées par l’agent virtuel sur l’utilisateur ? A partir de ces problématiques, nous avons axé nos travaux sur la modélisation informatique des émotions et sur leurs expressions faciales par un personnage virtuel réaliste. En effet, les expressions faciales sont une modalité privilégiée de la communication émotionnelle. Notre objectif principal est de contribuer l’amélioration de l’interaction entre l’utilisateur et un agent virtuel expressif. Nos études ont donc pour objectif de mettre en lumière les avantages et les inconvénients des différentes approches des émotions ainsi que des méthodes graphiques étudiées. Nous avons travaillé selon deux axes de recherches complémentaires. D’une part, nous avons exploré différentes approches des émotions (catégorielle, dimensionnelle, cognitive, et sociale). Pour chacune de ces approches, nous proposons un modèle informatique et une méthode d’animation faciale temps réel associée. Notre second axe de recherche porte sur l’apport du réalisme visuel et du niveau de détail graphique à l’expressivité de l’agent. Cet axe est complémentaire au premier, car un plus grand niveau de détail visuel pourrait permettre de mieux refléter la complexité du modèle émotionnel informatique utilisé. Les travaux que nous avons effectués selon ces deux axes ont été évalués par des études perceptives menées sur des utilisateurs.La combinaison de ces deux axes de recherche est rare dans les systèmes d’agents virtuels expressifs existants. Ainsi, nos travaux ouvrent des perspectives pour l’amélioration de la conception d’agents virtuels expressifs et de la qualité de l’interaction homme machine basée sur les agents virtuels expressifs interactifs. L’ensemble des logiciels que nous avons conçus forme notre plateforme d’agents virtuels MARC (Multimodal Affective and Reactive Characters). MARC a été utilisée dans des applications de natures diverses : jeu, intelligence ambiante, réalité virtuelle, applications thérapeutiques, performances artistiques, etc. / Emotions and their expressions by virtual characters are two important issues for future affective human-machine interfaces. Recent advances in psychology of emotions as well as recent progress in computer graphics allow us to animate virtual characters that are capable of expressing emotions in a realistic way through various modalities. Existing virtual agent systems are often limited in terms of underlying emotional models, visual realism, and real-time interaction capabilities. In our research, we focus on virtual agents capable of expressing emotions through facial expressions while interacting with the user. Our work raises several issues: How can we design computational models of emotions inspired by the different approaches to emotion in Psychology? What is the level of visual realism required for the agent to express emotions? How can we enable real-time interaction with a virtual agent? How can we evaluate the impact on the user of the emotions expressed by the virtual agent? Our work focuses on computational modeling of emotions inspired by psychological theories of emotion and emotional facial expressions by a realistic virtual character. Facial expressions are known to be a privileged emotional communication modality. Our main goal is to contribute to the improvement of the interaction between a user and an expressive virtual agent. For this purpose, our research highlights the pros and cons of different approaches to emotions and different computer graphics techniques. We worked in two complementary directions. First, we explored different approaches to emotions (categorical, dimensional, cognitive, and social). For each of these approaches, a computational model has been designed together with a method for real-time facial animation. Our second line of research focuses on the contribution of visual realism and the level of graphic detail of the expressiveness of the agent. This axis is complementary to the first one, because a greater level of visual detail could contribute to a better expression of the complexity of the underlying computational model of emotion. Our work along these two lines was evaluated by several user-based perceptual studies. The combination of these two lines of research is seldom in existing expressive virtual agents systems. Our work opens future directions for improving human-computer interaction based on expressive and interactive virtual agents. The software modules that we have designed are integrated into our platform MARC (Multimodal Affective and Reactive Characters). MARC has been used in various kinds of applications: games, ubiquitous intelligence, virtual reality, therapeutic applications, performance art, etc.

Context. This paper aims to investigate the potential of sustainable interaction system by employing persuasive design process. Sustainable Human Computer Interaction community (HCI) strives to find effective ways to change human behavior toward pro-environment. The sustainable HCI community seems to propose ambient display interface as one of its major interaction methods, which have an impact in quiet and static surroundings. However, when it comes to crowded public places where public resources are heavily wasted, ambient interfaces hardly get people’s attention and provide timely information to people. Goals. The author proposes an embodied sound interface as an alternative in the crowded places. The embodied sound interface serves to alert people at the right moment in a fun and pleasant way to induce people to engage in an intended behavior spontaneously. A sensor system prototype embedding an embodied sound interface, musical hand shaker (MHS), was developed aiming at reducing the use of paper towels. The MHS is placed near the water tap in a restroom and it respond with a music to the hand shaking of the user in front of the MHS. The system encourages people to experience how easy it is to save paper towel and do a green activity in their daily life. Methods. This study is grounded on three HCI fields: ambient information systems, affective interaction, and persuasive technology. The MHS prototype went through the persuasive design process which includes iterative steps of prototype implementation and evaluation. This design process centered on the eight steps of design process and Fogg behavior model. The persuasive qualities of embodied sound interface such as understandability, recognition of the system goal, and persistency of raised awareness of sustainability were examined. Results. The final outdoor evaluation in the restroom of a mall confirmed the impact of the MHS on people’s habit of using paper towels as well as its effectiveness of persuasion. Since it is a pilot study, there remain some unresolved issues and open questions for researcher in HCI and psychology fields.