This text is the result of a research project which began in summer 2015. I spent the past two years visiting various laboratories concerned with questions of life: the Pelling Lab, SymbioticA, the Chooi Lab and the HumAnimaLab. My methods have been highly immersive and at the edges of autoethnography. I have navigated gestures and a cellular anthropology to gain a better understanding of the relations at play within the laboratories I have grown with and learned from. Interconnected moving livings is what I stumbled upon in these spaces of scientific, artistic and, most importantly, embodied exploration. By characterizing these specific biotechnological relations and mediations which are in processes of articulation, I explore the notion of crafting. I draw from the literatures of the anthropology of life, anthropology of craft as well as from craft theory to speak of concurrent laboratory livings as engaging in a crafting with livings.

Gestures are common in baroque opera performances, to help communicate the content and is a way to be more expressive with the whole body while performing. I wanted to explore baroque gestures to improve and expand my capacity to communicate through expressive movement, to develop my own skills as an opera singer and to add another important layer to my baroque performance. I have analysed La Messaggiera’s aria, Un in fiorito prato, from Claudio Monteverdi’s opera L’Ofreo. This by analysing the character, the aria and the music, including a poetic and word by word translation. I chose words from the aria and combined them with gestures. Overall, three recordings were made; one baseline, to show how I usually perform and two recordings applying and developing the gestures. The second and the third recordings were evaluated by myself and three separate reviewers, by comparing the recordings and providing feedback on my baroque gesture performance development. The feedback stated that the applied gestures helped to convey the dramatic content and enhanced my sense of phrasing and helped to reinforce the soft and loud contrasts of the singing. This helped to enhance the reviewers’ perceptions of the character’s emotional expression of the aria. I will continue to develop my knowledge and use of baroque gestures when performing baroque opera repertoire.

Presence, as a perceptual illusion of non-mediation, is a phenomenon that is often vaguely conceptualized, measured via self-report and stimulated via technological factors. This study offers a more certain conceptual framework, behavioral measure, and a contextual factor that highlights the psychological nature of this psychological state. Specifically, a Zoom-based field experiment was conducted to test whether being visually familiarized with a person before describing spatial information (images of an old dress and maze) to another person would increase a sense of social presence, and therefore a higher rate of gestures used to describe images even though they won’t be seen by the other person. As results showed, being familiarized was enough to reduce uncertainty over the other person to feel social presence and gesture at a higher rate as if the communication was face-to-face. / Media & Communication

<p>This research focuses on making animated pedagogical agents more emotionally expressive to facilitate students’ learning process and create a true connection between the agents and the students for effective learning. The author of this thesis focused on applying affective hand and arm gestures that were derived from the Geneva Multimodal Emotion Portrayals (GEMEP) core set of affective expressions on top of animated lecturing agents. The GEMEP database includes 145 video recordings of French actors portraying different emotions. Only arm and hand gestures were extracted from the database and were applied on top of lecturing animations of animated pedagogical agents. 131 participants participated in an online perception study where they were asked to watch different animations of six basic emotions (anger, joy, sadness, disgust, fear, and surprise) displayed by male and female characters. Results of the perception study showed that only one angry expression by the female character passed the 75% bar of recognition rate, while the remaining expressions showed a low recognition rate (1.5% -64%). The author also examined whether levels of emotion arousal and valence were correctly identified by the students. Participants’ identification of correct levels of arousal and valence was more accurate than recognition of Ekman’s basic emotions but was still not sufficient for further statistical analysis on validating hypotheses of research on observing how gender of participants and characters influence the emotion perception. Overall, significant reasons were identified that influenced the low perception rate among participants, which are discussed in the conclusion. </p>

Studien syftar till att undersöka sambandet mellan tal och gester utifrån lärares resonemang och språkbruk i klassrummet. Studien utgår från ett förkroppsligande perspektiv där begreppen deiktiska gester, metaforiska gester, ikoniska gester och rytmiska gester återfinns. Begreppen är hämtade från McNeills (1992) indelning av gester i olika kategorier. För insamling av data användes ostrukturerad observation och semistrukturerad intervju. Resultatet från observationerna visade att deiktiska rörelser, främst att peka mot tavlan, var den mest förekommande gesten hos lärarna och de rytmiska gesterna förekom inte alls. De näst mest förekommande gesterna var metaforiska gester som symboliserar en abstrakt idé, någon handling som utförs. Genom intervjuerna framkom att lärarna ansåg det viktigt att använda gester i undervisningen. Studiens didaktiska implikation blir att gesterna tillför liv och rörelse i undervisningen och är en bra hjälp för att förtydliga ord och begrepp både för elever med svenska som första språk och elever med svenska som andraspråk. Av studiens resultat dras slutsatsen att gester och verbal kommunikation är en användbar kombination i undervisningen för att eleverna ska ta till sig budskapet.

This thesis concerns the formulation of novel developmental robotics models of embodied phenomena in number learning. Learning to count is believed to be of paramount importance for the acquisition of the remarkable fluency with which humans are able to manipulate numbers and other abstract concepts derived from them later in life. The ever-increasing amount of evidence for the embodied nature of human mathematical thinking suggests that the investigation of numerical cognition with the use of robotic cognitive models has a high potential of contributing toward the better understanding of the involved mechanisms. This thesis focuses on two particular groups of embodied effects tightly linked with learning to count. The first considered phenomenon is the contribution of the counting gestures to the counting accuracy of young children during the period of their acquisition of the skill. The second phenomenon, which arises over a longer time scale, is the human tendency to internally associate numbers with space that results, among others, in the widely-studied SNARC effect. The PhD research contributes to the knowledge in the subject by formulating novel neuro-robotic cognitive models of these phenomena, and by employing these in two series of simulation experiments. In the context of the counting gestures the simulations provide evidence for the importance of learning the number words prior to learning to count, for the usefulness of the proprioceptive information connected with gestures to improving counting accuracy, and for the significance of the spatial correspondence between the indicative acts and the objects being enumerated. In the context of the model of spatial-numerical associations the simulations demonstrate for the first time that these may arise as a consequence of the consistent spatial biases present when children are learning to count. Finally, based on the experience gathered throughout both modelling experiments, specific guidelines concerning future efforts in the application of robotic modelling in mathematical cognition are formulated.

This thesis will explore the function of the conductor’s left hand, with emphasis on its expressive possibilities and the gestures. In the first chapter, the role of the conductor is discussed, and some problems around the field of expressive gestures are brought up. In the second chapter, I present the history of conducting, or rather the history of musical leadership, as the practice of conducting we are familiar with today began in the late 19th Century. In this chapter, I also include some paragraphs about gesture and its role and function outside of musical leadership, such as public speaking. In chapter three, I examine a selection of instruction books and how they treat the use of the left hand. During the work on this material I found several quotations and thoughts from conductors, in letters and interviews, presented in chapter four. This is interesting for this study, because it offers the personal opinions and experiences of well-established conductors and conducting teachers. It also shows how the opinions on the use of the left hand have changed over the past century. Chapter five is a lengthy chapter of research on this topic. Not all is directly connected to the left hand, but all is in connection to gesture in some way. These chapters form the basis of a discussion, where findings in previous chapters are pointed out and debated, and the authors personal opinion is presented. At the end follows suggestions on how this topic can be further researched and how one can create a repertoire of expressive gestures.

Biometrics are a convenient alternative to traditional forms of access control such as passwords and pass-cards since they rely solely on user-specific traits. Unlike alphanumeric passwords, biometrics cannot be given or told to another person, and unlike pass-cards, are always “on-hand.” Perhaps the most well-known biometrics with these properties are: face, speech, iris, and gait. This dissertation proposes a new biometric modality: gestures. A gesture is a short body motion that contains static anatomical information and changing behavioral (dynamic) information. This work considers both full-body gestures such as a large wave of the arms, and hand gestures such as a subtle curl of the fingers and palm. For access control, a specific gesture can be selected as a “password” and used for identification and authentication of a user. If this particular motion were somehow compromised, a user could readily select a new motion as a “password,” effectively changing and renewing the behavioral aspect of the biometric. This thesis describes a novel framework for acquiring, representing, and evaluating gesture passwords for the purpose of general access control. The framework uses depth sensors, such as the Kinect, to record gesture information from which depth maps or pose features are estimated. First, various distance measures, such as the log-euclidean distance between feature covariance matrices and distances based on feature sequence alignment via dynamic time warping, are used to compare two gestures, and train a classifier to either authenticate or identify a user. In authentication, this framework yields an equal error rate on the order of 1-2% for body and hand gestures in non-adversarial scenarios. Next, through a novel decomposition of gestures into posture, build, and dynamic components, the relative importance of each component is studied. The dynamic portion of a gesture is shown to have the largest impact on biometric performance with its removal causing a significant increase in error. In addition, the effects of two types of threats are investigated: one due to self-induced degradations (personal effects and the passage of time) and the other due to spoof attacks. For body gestures, both spoof attacks (with only the dynamic component) and self-induced degradations increase the equal error rate as expected. Further, the benefits of adding additional sensor viewpoints to this modality are empirically evaluated. Finally, a novel framework that leverages deep convolutional neural networks for learning a user-specific “style” representation from a set of known gestures is proposed and compared to a similar representation for gesture recognition. This deep convolutional neural network yields significantly improved performance over prior methods. A byproduct of this work is the creation and release of multiple publicly available, user-centric (as opposed to gesture-centric) datasets based on both body and hand gestures.

O presente estudo aborda o papel da gesticulação espontânea no ensino-aprendizagem de língua estrangeira. Verificamos em experimento o impacto da gesticulação do narrador na compreensão de narrativas por ouvintes aprendizes de língua estrangeira. Controlamos a presença e ausência do fator gestual na apresentação das narrativas. A presença de gestos aumentou o desempenho dos voluntários na tarefa de compreensão. Para entendermos melhor a relação entre gesticulação e compreensão de narrativas, correlacionamos dados de desempenho dos voluntários com medidas comportamentais de movimentação e fixação oculares registradas durante a realização dos experimentos por equipamento de rastreamento ocular / The present study investigates the role of spontaneous gesticulation in the process of teaching and learning a foreign language. We studied experimentally the impact of the narrators gesticulation on the comprehension of narratives by students of English as a foreign language. We controlled the presence and absence of hand gesticulation in the presentation of the stimuli. The presence of gestures increased volunteers performance in the comprehension task. For a better understanding of the relation between gesticulation and comprehension of narratives, we correlated data from volunteers perfomance with behavioral measures of eye movement and fixation, captured during the experiment by eyetracking methodology

The fields of virtual reality and gesture-based input devices are growing and becoming more popular. In order for the two technologies to be implemented together, an understanding of gestures in relation to virtual objects and users' expectations of those gestures needs to be understood. Specifically, this thesis focuses on arm gestures for the rotation of virtual objects. Participants in the study were first asked to freely perform an arm gesture they felt should execute a task. Next, participants were asked to perform specific rotation tasks with pre- configured arm gestures on four objects. There were two types of objects: those that could only be rotated on one axis and those that could be rotated on two axes. Each object type was represented by a familiar small and large object: combination lock, water wheel, baseball and beach ball. Data on how quickly they could complete the rotation tasks was collected. After performing the tasks on each of the four objects, participants were asked to rate the intuitiveness of each gesture as well as their preferred gesture for the specific task. The captured data showed that when users were presented with virtual representations of familiar physical objects, most of them expected to rotate the objects with the same gestures they would use on the actual physical objects. Considering 1-axis objects, an arm-based twist gesture outperformed other arm-based gestures in both intuitiveness and efficiency. Also with 2-axis objects, an arm-based horizontal/vertical gesture outperformed others in both intuitiveness and efficiency. Interestingly, those gestures were most efficient for each object type regardless of the size of the object being rotated. This would indicate that users are able to mentally separate the physical and virtual experiences. Larger objects require different rotation gestures than smaller objects in the physical world, but that requirement is non-existent in a virtual world. However, while the mind can separate between the physical and virtual worlds, there is still an expected connection. This is based on the fact that the gestures most preferred for the rotation tasks are the same gestures used for similar physical tasks.