To understand what modality affects the perception of emotion is interesting since it would help us get closer to understanding and dissecting emotions. Earlier research has shown that body motions can help disambiguate ambiguous (determining something vaguely presented) facial expressions but there is no study that examines the possibility of facial expressions to affect the perception of body motion. That is why this study aims to examine if facial expressions can affect the perception of full body emotions. This is done with a perceptual experiment in which 22 subjects were exposed to stimuli consisting of scenes with virtual characters expressing emotions (see figure 1). It was concluded that the facial expression does affect the perception of joined valence within the group of characters and it is thereby proven that not only body motions can help disambiguating ambiguous facial expressions but that facial expressions can help disambiguating ambiguous body motions and alter the perception of distinct body motions perceived as either positive or negative without face and hands showing. It is also concluded that perceived trustworthiness is affected by the valence of the facial expression, which supports recent studies conducted which aimed to find out what makes for a trustworthy and dominant face with the help of valence as a factor. The perceived relationship within the group is also examined as well as the dominance of the group as a whole but neither gave results clear enough to make any conclusions except that positive valence affects the perceived relationship within the group as closer. The study is conducted using virtual agents but is meant to help better understand people in everyday situations.

L’utilisation de personnages virtuels dans le cadre de simulations basées sur les lois de la physique trouve maintenant des applications allant de la biomécanique à l’animation. L’un des éléments incontournables de cette performance est le contrôleur de mouvement, capable de transformer les actions souhaitées en mouvements synthétisés. La conceptualisation de ces contrôleurs a profondément évolué grâce à l'apport des connaissances en biomécanique qui a conduit à l'utilisation de modèles de personnages encore plus détaillés car s'inspirant de l’appareil squelettique et surtout musculaire de l’être humain (ou personnages à modèle musculaire). Contrôler les personnages virtuels implique un défi de taille : contrôler la redondance, ou le fait même qu’un nombre important de muscles ou d’actionneurs aient besoin d’être contrôlés simultanément pour exécuter la tâche de motricité demandée.L’objectif de cette thèse est d’y répondre en s’inspirant du système de contrôle moteur humain permettant de gérer cette redondance. Une solution de contrôle, pour les personnages virtuels, est proposée d’après la théorie des synergies musculaires et appliquée à des mouvements de contrôle du lancer. Les synergies musculaires sont des représentations de contrôle à faible dimension et qui permettent aux muscles d’être contrôlés en groupe, réduisant ainsi de manière significative le nombre de variables. Grâce à cette stratégie, cette thèse permet les contributions suivantes : en premier lieu, la validation de la théorie des synergies musculaires, utilisée ici pour étudier un nouveau mouvement et pour tenter de contrôler un personnage virtuel. Et elle contribue également à l'ensemble des domaines impliquant des simulations corporelles, ayant recours aux personnages à modèle musculaire (comme par exemple, la biomécanique ou l'animation) en leur proposant une solution de contrôle permettant de réduire la redondance. / The use of virtual characters in physics-based simulations has applications that range from biomechanics to animation. An essential component behind such applications is the character’s motion controller, which transforms desired tasks into synthesized motions. The way these controllers are designed is being profoundly transformed through the integration of knowledge from biomechanics, which motivates the idea of using more detailed character models, inspired by the human musculoskeletal system (or muscle-based characters). Controlling these characters implies solving an important challenge: control redundancy, or the fact that numerous muscles or actuators need to be coordinated simultaneously to achieve the desired motion task.The goal of this thesis is to address this challenge by taking inspiration from how the human motor control system manages this redundancy. A control solution for virtual characters is proposed based on the theory of muscle synergies, and applied on the control of throwing motions. Muscle synergies are low-dimensional control representations that allow muscles to be controlled in groups, thus reducing significantly the number of control variables.Through this solution this thesis has the following contributions: 1) A contribution to the validation of the muscle synergy theory by using it to study a new motion, and challenging it with the control of a virtual character, and 2) a contribution to the variety of domains involving physical simulation with muscle-based characters (e.g, biomechanics, animation) by proposing a control solution that reduces redundancy.

Submitted by MAÍRA CASTILHOS COELHO null (mcastilhos@gmail.com) on 2017-09-18T18:49:59Z No. of bitstreams: 1 MairaCastilhosCoelho_Tese_2017.pdf: 9546799 bytes, checksum: 1f56dee4e10ca12bc631066c68236a64 (MD5) / Approved for entry into archive by Monique Sasaki (sayumi_sasaki@hotmail.com) on 2017-09-19T20:17:19Z (GMT) No. of bitstreams: 1 coelho_mc_dr_ia.pdf: 9546799 bytes, checksum: 1f56dee4e10ca12bc631066c68236a64 (MD5) / Made available in DSpace on 2017-09-19T20:17:19Z (GMT). No. of bitstreams: 1 coelho_mc_dr_ia.pdf: 9546799 bytes, checksum: 1f56dee4e10ca12bc631066c68236a64 (MD5) Previous issue date: 2017-07-24 / Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) / A pesquisa aqui proposta pretende refletir sobre as formas de presença do ator existentes no teatro a partir da fusão com a linguagem cinematográfica. Sob o ponto de vista do ator, foi pesquisada a nova dimensão da sua presença no palco, caracterizada pela utilização da câmara e do microfone. Para tanto, introduzimos uma breve história do teatro durante o surgimento do cinema visando contextualizar os estudos e reflexões contidos nos capítulos seguintes, nos quais serão aprofundados os avanços tecnológicos que culminaram no momento presente. Ao estudar as possibilidades do uso de recursos digitais no teatro, o foco é direcionado para as relações entre ator, câmera e projeção, visando investigar as possíveis formas de presença que vem sendo exploradas no teatro. Assim, o estudo é desenvolvido a partir das encenações: "E se elas fossem para Moscou?", de Christiane Jatahy; "King of War", de Ivo Van Hove; " Le Projet Andersen", de Robert Lepage; "The Tempest", de 4DArt; "Eraritjaritjaka", de Heiner Goebbels. A escolha dos espetáculos se deu devido a diferentes formas de captação e projeção das imagens em vídeo, que possibilitam, formas distintas de presença e de convívio entre ator-câmera-espectador. Por fim, é esperado, também, que o estudo propicie subsídios analíticos para a investigação das novas possibilidades deste “teatrodigital” e das tendências que se desanuviam em um futuro próximo de evolução tecnológica e diversificação técnica. / The present research intends to reflect on actor’s types of presence in theatre from the fusion with the cinematographic language. Under the actor’s point of view, it was researched a new dimension of his presence on stage, characterized by the use of camera and microphone. To this end, the study begins with a short history of theatre during the rise of cinema so as to contextualize the studies and reflections contained in future chapters, which present technological advances culminating in the present day. In studying the possibilities of this “theatre-cinema”, the research focuses on relationships between actors, cameras, and projections, as well as investigating possible forms of presence that both cinema and theatre explore, and that enable new relations between actors and spectators to be formed. The study examines the following scenic productions: "E se elas fossem para Moscou?" by Christiane Jatahy; "King of War", by Ivo Van Hove; "Le Projet Andersen", by Robert Lepage; "The Tempest", by 4DArt; and "Eraritjaritjaka", by Heiner Goebbels. The choice of productions is based on different forms of capturing and projecting video images, which allow for distinct modes of presence and of interaction between actors, cameras, and spectators. Finally, the research also aims to supply analytic tools toward the investigation of new possibilities of this “theatre-cinema” and of tendencies that will arise in the near future, based on technological evolution and technical diversification. / FAPESP: 2013/06541-3 / FAPESP: 2015/06139-6

La communication multimodale, qui est primordiale dans les relations interpersonnelles, reste encore très limitée dans les interfaces homme-machine actuelles. Parmi les différentes modalités qui ont été adoptées par les recherches en interaction homme-machine, la modalité posturale a été moins explorée que d’autres modalités comme la parole ou les expressions faciales. Les postures corporelles sont pourtant indispensables pour interpréter et situer l’interaction entre deux personnes, que ce soit en termes de contexte spatial ou de contexte social. Les briques de base que sont les dispositifs et les modalités d’interaction (par exemple la kinect ou les avatars) sont pourtant disponibles. Il manque cependant des modèles informatiques reliant ces médias et modalités aux fonctions de communication pertinentes dans les interactions interpersonnelles comme celles liées à l’espace ou aux émotions.L’objectif de cette thèse est de concevoir un premier modèle informatique permettant d’exploiter les postures dans les interactions homme-machine. Cela soulève plusieurs questions de recherche : comment représenter symboliquement des postures exprimées durant des interactions interpersonnelles ? Comment spécifier les comportements posturaux de personnages virtuels ? Quelles doivent être les caractéristiques d’un modèle d’interaction corporel permettant des interactions entre un personnage virtuel et un utilisateur dans différents contextes physiques ou sociaux ?L’approche proposée consiste dans un premier temps à prendre comme point de départ des corpus vidéo filmés dans différentes situations. Nous avons défini un schéma de codage pour annoter manuellement les informations posturales à différents niveaux d’abstraction et pour les différentes parties du corps. Ces représentations symboliques ont été exploitées pour effectuer des analyses des relations spatiales et temporelles entre les postures exprimées par deux interlocuteurs.Ces représentations symboliques de postures ont été utilisées dans un deuxième temps pour simuler des expressions corporelles de personnages virtuels. Nous nous sommes intéressés à un composant des émotions particulièrement pertinent pour les études sur les postures : les tendances à l’action. Des animations impliquant deux personnages virtuels ont été ainsi conçues puis évaluées perceptivement.Enfin, dans un troisième temps, les expressions corporelles d’un personnage virtuel ont été conçues dans une application mixte faisant intervenir un personnage virtuel et un utilisateur dans un cadre d’interaction ambiante. Les postures et gestes du personnage virtuel ont été utilisées pour aider l’utilisateur à localiser des objets du monde réel.Cette thèse ouvre des perspectives sur des études plus spécifiques de l’interaction corporelle nécessitant par exemple des annotations automatiques via des dispositifs de capture de mouvement ou la prise en compte des différences individuelles dans l’expression posturale. / Multimodal communication is key to smooth interactions between people. However, multimodality remains limited in current human-computer interfaces. For example, posture is less explored than other modalities, such as speech and facial expressions. The postural expressions of others have a huge impact on how we situate and interpret an interaction. Devices and interfaces for representing full-body interaction are available (e.g., Kinect and full-body avatars), but systems still lack computational models relating these modalities to spatial and emotional communicative functions.The goal of this thesis is to lay the foundation for computational models that enable better use of posture in human-computer interaction. This necessitates addressing several research questions: How can we symbolically represent postures used in interpersonal communication? How can these representations inform the design of virtual characters' postural expressions? What are the requirements of a model of postural interaction for application to interactive virtual characters? How can this model be applied in different spatial and social contexts?In our approach, we start with the manual annotation of video corpora featuring postural expressions. We define a coding scheme for the manual annotation of posture at several levels of abstraction and for different body parts. These representations were used for analyzing the spatial and temporal relations between postures displayed by two human interlocutors during spontaneous conversations.Next, representations were used to inform the design of postural expressions displayed by virtual characters. For studying postural expressions, we selected one promising, relevant component of emotions: the action tendency. Animations were designed featuring action tendencies in a female character. These animations were used as a social context in perception tests.Finally, postural expressions were designed for a virtual character used in an ambient interaction system. These postural and spatial behaviors were used to help users locate real objects in an intelligent room (iRoom). The impact of these bodily expressions on the user¡¯s performance, subjective perception and behavior was evaluated in a user studyFurther studies of bodily interaction are called for involving, for example, motion-capture techniques, integration with other spatial modalities such as gaze, and consideration of individual differences in bodily interaction.

Multimodal communication is key to smooth interactions between people. However, multimodality remains limited in current human-computer interfaces. For example, posture is less explored than other modalities, such as speech and facial expressions. The postural expressions of others have a huge impact on how we situate and interpret an interaction. Devices and interfaces for representing full-body interaction are available (e.g., Kinect and full-body avatars), but systems still lack computational models relating these modalities to spatial and emotional communicative functions.The goal of this thesis is to lay the foundation for computational models that enable better use of posture in human-computer interaction. This necessitates addressing several research questions: How can we symbolically represent postures used in interpersonal communication? How can these representations inform the design of virtual characters' postural expressions? What are the requirements of a model of postural interaction for application to interactive virtual characters? How can this model be applied in different spatial and social contexts?In our approach, we start with the manual annotation of video corpora featuring postural expressions. We define a coding scheme for the manual annotation of posture at several levels of abstraction and for different body parts. These representations were used for analyzing the spatial and temporal relations between postures displayed by two human interlocutors during spontaneous conversations.Next, representations were used to inform the design of postural expressions displayed by virtual characters. For studying postural expressions, we selected one promising, relevant component of emotions: the action tendency. Animations were designed featuring action tendencies in a female character. These animations were used as a social context in perception tests.Finally, postural expressions were designed for a virtual character used in an ambient interaction system. These postural and spatial behaviors were used to help users locate real objects in an intelligent room (iRoom). The impact of these bodily expressions on the user¡¯s performance, subjective perception and behavior was evaluated in a user studyFurther studies of bodily interaction are called for involving, for example, motion-capture techniques, integration with other spatial modalities such as gaze, and consideration of individual differences in bodily interaction.

VIEIRA, Roberto Cesar Cavalcante. Reprodução e manipulação de personagens virtuais. 2012. 173 f. Tese (Doutorado em ciência da computação)- Universidade Federal do Ceará, Fortaleza-CE, 2012. / Submitted by Elineudson Ribeiro (elineudsonr@gmail.com) on 2016-07-20T12:58:57Z No. of bitstreams: 1 2012_tese_rccvieira.pdf: 30143588 bytes, checksum: 511b1ee55b952226f7674aab036f2abb (MD5) / Approved for entry into archive by Rocilda Sales (rocilda@ufc.br) on 2016-07-25T12:10:42Z (GMT) No. of bitstreams: 1 2012_tese_rccvieira.pdf: 30143588 bytes, checksum: 511b1ee55b952226f7674aab036f2abb (MD5) / Made available in DSpace on 2016-07-25T12:10:42Z (GMT). No. of bitstreams: 1 2012_tese_rccvieira.pdf: 30143588 bytes, checksum: 511b1ee55b952226f7674aab036f2abb (MD5) Previous issue date: 2012 / Many virtual reality applications and games need a large number of virtual characters. Some of these applications require, in addition to quantity, the simulation of kinship and evolution, not only of human character models but also of di erent types of animals, toon models or other creatures. Some applications also require interactions between isolated populations with well-de ned ethnic characteristics. The identi cation of similar traits between individuals of the same family is crucial to providing increased realism to many of these systems. The main di culty in these situations is to generate models automatically, in real time, which are physically similar to a given population or family. Other realistic desirable features are the automatic body variations due to epigenetic factors and the application of facial expressions as the character interacts with the environment. In those cases, the di culty lies in nding a simple mesh adaptation system for di erent creatures with big di erences in shape. In this work, the reproduction of diploid beings is mimicked to produce character models that inherit genetic characteristics from their ancestors, with the possibility to map all genes identifying the origin of each gene. Unlike morphing techniques and other existing approaches, in our method, it is possible for a genetic characteristic from an ancestor to be manifested only after a few generations. After character generation, it is possible to apply custom body variations and facial expressions to the new models. With the same adaptation system used in all methods, it is possible to generate caricatures of characters, inserting a comic atmosphere to the application. With this solution it is possible to create interactive evolution and life-simulation games, genetics educational applications, and many other possibilities. / Muitas aplicações de realidade virtual e jogos necessitam de um grande número de personagens virtuais. Algumas dessas aplicações requerem, além da quantidade, a simulação de parentesco e evolução, não só de modelos de personagens humanos, mas também de diferentes tipos de animais, modelos caricaturados ou outras criaturas. Algumas aplicações também requerem interações entre populações isoladas, com características étnicas bem definidas. A identificação de características semelhantes entre os indivíduos de uma mesma família é fundamental para proporcionar maior realismo a muitos desses sistemas. A principal dificuldade nessas situações é gerar modelos automaticamente, em tempo real, que são fisicamente semelhantes a uma dada população ou família. Outra característica de realismo desejável nessas aplicações seria a possibilidade de variações automáticas do corpo por fatores epigenéticos e aplicação de expressões faciais à medida que o personagem interage com o ambiente. Nesses casos, a dificuldade reside em encontrar um sistema simples de adaptação de malha para criaturas com grandes diferenças na forma. Nesse trabalho, a reprodução de seres diploides é simulada para produzir modelos de personagens que herdam características dos seus ancestrais, com a possibilidade de mapear todos os genes, identificando a origem de cada um deles, e aplicar variações corporais e expressões faciais aos novos modelos. Com o mesmo sistema de adaptação utilizado em todos os métodos é possível gerar caricaturas de personagens, inserindo uma atmosfera de humor à aplicação. Com esta solução é possível criar jogos interativos de evolução e simulação de vida, aplicações educativas de genética e muitas outras possibilidades. Ao contrário de técnicas de morphing e de outros enfoques existentes, no método aqui proposto, é possível que uma característica genética de um ancestral se manifeste somente depois de algumas gerações.

How is it possible to design virtual characters that can express different emotions such as compound emotions that are a mix of basic emotion expressions? Augmented reality (AR) can create engaging experiences for participants, and in recent years, virtual faces and virtual characters have become increasingly realistic and expressive, for example, when reducing costs using therapeutic applications. The validity of virtual expression has been shown in studies on desktop computers but less so in AR. In this paper, the basic emotions and the compound emotions of virtual characters were studied and the character was designed to work with Microsoft HoloLens in AR. The process of creating basic emotion blends to a human virtual character was created and the animations were modified using Unity3D game engine. The participants (n = 24) experienced the virtual character in a job interview context wearing HoloLens mixed reality glasses. The virtual character made basic and compound facial expressions and the participants were asked to label them. The result show that all participants successfully recognized seven basic emotions and seven compound emotions from a virtual character in AR using HoloLens; however, disgust was confused with sad, and angry was sometimes confused with disgust. Also, the fearfully surprised was often mistaken with awed. The result show that the compound emotions were recognized quite well and the results indicate that the perceived valence changes depending on the facial expressions. The study provides insights into how blended emotive expressions for virtual characters are created and perceived. / Hur är det möjligt att designa virtuella karaktärer som kan uttrycka olika känslor, till exempel blandade känslor som är en blandning av grundläggande känslor? Augmented Reality (AR) kan skapa engagerande upplevelser för deltagarna, och de senaste åren har virtuella ansikten och virtuella karaktärer blivit alltmer realistiska och uttrycksfulla, till exempel då kostnaderna minskas med hjälp av terapeutiska tillämpningar. Giltigheten av virtuellt uttryck har visats i studier på stationära datorer, men mindre i AR. I denna rapport studerades de grundläggande känslorna och hur de blandade känslorna av virtuella karaktären kan konstrueras för att fungera med Microsoft HoloLens i AR. Processen att skapa blandade känslor till en mänsklig virtuell karaktär konstruerades och animationerna modifierades med hjälp av Unity3D-spelmotor. Deltagarna (n = 24) upplevde den virtuella karaktären i en jobbintervju med Microsofts HoloLens. Den virtuella karaktären gjorde grundläggande och sammansatta ansiktsuttryck och deltagarna ombads kategorisera dom. Resultatet visar att alla deltagare framgångsrikt erkänt sju grundläggande känslor och sju blandade känslor från en virtuell karaktär i AR med hjälp av HoloLens; disgust var ihopblandad med sad, och angry var ibland ihopblandad med disgust. Fearfully surprised blev också ofta felaktig ihopblandad med awed. Studiens resultat visar att de blandade känslorna var lätta att känna igen och resultaten antyder på att den uppfattade valens förändras beroende på ansiktsuttryck. Studien ger insikter om hur blandade känslouttryck för virtuella karaktärer har konstruerats och uppfattats.

La pléthymographie pénienne est considérée comme la méthode la plus objective pour évaluer les intérêts sexuels d'hommes adultes. Par contre, un nombre grandissant de chercheurs déplorent le manque de correspondance qui existe entre les stimuli présentés dans ce type d'évaluation et la réalité. Il est proposé de tenter de remédier à cette lacune en utilisant des personnages virtuels générés par ordinateur et présentant une variété de dispositions affectives lors d'évaluations pléthysmographiques. Ces stimuli n'ont jamais été utilisés dans l'évaluation des intérêts sexuels auprès d’agresseurs sexuels d'enfants. Cette thèse est composée de deux articles qui visent à soutenir le recours à ce type de stimuli lors d'évaluations pléthysmographiques. Afin d'atteindre cet objectif, des hommes ayant eu ou non des contacts sexuels avec des enfants ont été recrutés dans le but de les comparer. Une première étude empirique est bipartite. Elle vise en premier lieu à déterminer auprès d'une population étudiante si les dispositions affectives simulées par les personnages virtuels sont identifiables. Elle cherche ensuite à déterminer la capacité des personnages virtuels à générer un profil d'excitation sexuelle propre à un groupe d'individus n'ayant pas de problématique sexuelle. La seconde étude expérimentale porte sur la comparaison des profils de réponses érectiles issus de la présentation des personnages virtuels et évalue leur capacité à discriminer les individus selon qu'ils possèdent ou non des antécédents en matière de délinquance sexuelle envers les enfants. Dans l'ensemble, les résultats soutiennent l'utilisation de ces stimuli lors d'évaluations pléthysmographiques. Ils démontrent que les personnages virtuels simulent les dispositions affectives prévues et qu'ils génèrent des profils d'excitation sexuelle et des indices de déviance permettant de discriminer entre les groupes à l'étude. Ce projet de recherche présente les avantages associés à la présentation des personnages virtuels simulant différentes dispositions affectives lors de l'évaluation des intérêts sexuels chez les agresseurs d'enfants et propose des avenues intéressantes en termes d'évaluation et de traitement auprès de cette population. / Penile plethysmography is considered the most objective method to assess adult men's sexual interests. However, a growing number of researchers underline the poor correspondence between the stimuli used during this type of assessment and reality. It is proposed to try to overcome this issue by using computer generated virtual characters that present a variety of affective dispositions during plethysmographic assessments. These stimuli have never been used in the assessment of child molesters' sexual interests. This thesis is composed of two articles that aim at supporting the use of this type of stimuli during plethysmographic assessments. In order to reach this goal, men who have or not engaged in sexual contact with children were recruited so that they could be compared. A first empirical study is bipartite. On one hand, it aims to determine in a student sample whether the affective dispositions simulated by the virtual characters represent what was intended. Its second goal is to determine the ability of the charcarters to generate a sexual arousal profile representative of non-deviant men. The second experimental study focuses on comparing profiles of erectile responses recorded to the virtual characters and determining their ability to discriminate individuals according to their group membership. Overall, results support the use of these stimuli during plethysmographic assessments. They demonstrate that the virtual characters simulate intended affective dispositions as well as their ability to generate profiles of sexual arousal and deviance differentials representative of group membership. This research project presents the advantages of using virtual characters that simulate varying affective dispositions in the assessment of child molesters' sexual interests and proposes interesting avenues in terms of assessment and treatment with this population.

Denna studie undersöker hur spelupplevelsen i ett "management"-spel förändras om virtuella karaktärer agerar verklighetstroget i sina handlingar och val samt hur en spelutvecklare kan hålla nere påverkan på spelets prestanda och samtidigt bibehålla trovärdigheten hos spelets virtuella karaktärer. Resultat från tidigare forskning om AI-LOD har använts som grund i utvecklingen av en prototyp av ett "management"-spel som sedan testats av åtta deltagare följt av individuella intervjuer gällande deras upplevelser av och reflektioner kring spelet. Studien presenterar förslag på tillvägagångssätt för att använda AI-LOD effektivt samt hur användarnas upplevelser har påverkats positivt och negativt av de designval som gjorts. / This study investigates how the player experience is affected in "management" games when the virtual characters' behaviors are pivoted towards realism. The study also investigates the usage of AI-LOD as a method to reduce the impact on performance by these virtual characters without sacrificing authenticity of their behavior. Results and findings from previous research in this particular field are used as a foundation in the development of a prototype of a "management" game. This prototype has been tested by eight participants followed by individual interviews on their experiences and thoughts on the game. The study presents suggestions on how to use AI-LOD effectively and how player experience has been affected both positively and negatively by design choices made during the development of this prototype.