A comunicação humana é essencial para que ocorra um processo de interação social. É através dela que os sujeitos podem manifestar suas vontades, desejos, necessidades, estabelecer trocas que resultam em processos de ensino-aprendizagem e participar ativamente da sociedade, comunidade na qual se encontra inserido realizando trocas recíprocas que propiciam a aquisição do conhecimento historicamente construído. Esta pesquisa teve como foco principal investigar de que de que forma o uso de um sistema de CA que parte de uma perspectiva metodológica sócio-histórica pode promover o desenvolvimento de gestos que propiciam intencionalidade comunicativa em crianças de 03 a 05 anos com autismo. A presente pesquisa cunhou-se como um estudo de caso múltiplo de três sujeitos com autismo, não oralizados, com idades entre 03 e 05 anos de idade. Os resultados a serem apresentados mostram que o sistema SCALA contribuiu como forma constante de mediação integrada ao todo do sujeito. Dessa forma foram surgindo novos gestos, fomos significando cada olhar, apontar, sorrir, morder, etc. Considerando nossos sujeitos como agentes intencionais, deixando-os serem atores nas interações, não somente interagentes passivos, ancorados pelo uso de materiais concretos, que levaram a ligação entre o representacional e o simbólico e, sobretudo nos conduziram a diversos e diferentes momentos de atenção e cenas de atenção conjunta, onde nossos sujeitos participavam de interações triádicas e se incluíam nelas. / Human communication is essential to the occurrence of a social interaction process. It is through it that subjects can express their wishes, desires, needs and establish exchanges that result in teaching-learning processes. And, also, actively participate in society, the community in which they are inserted, conducting reciprocal exchanges that foster the acquisition of historically constructed knowledge. This research main focus was to investigate how the use of a CA system that starts from a sociohistorical methodological perspective can promote the development of gestures that provide communicative intentionality in children 03-05 years old with autism. This research was coined as a multiple case study of three subjects with autism, who do not use oral language, aged between 03 and 05 years of age. The results to be presented show that the SCALA system contributed steadily as integrated mediation into the subject as a whole. Thus new gestures started to arise, we gave meaning to each look, point, laugh, biting, etc. Considering our subjects as intentional agents, leaving them to be actors in interactions, not only passive interacting agents, anchored by the use of concrete materials, which led the connection between the representational and the symbolic and above all led us to several different moments of attention and joint attention scenes where our subjects participated in triadic interactions and included themselves in these interactions.

Submitted by Irene Nascimento (irene.kessia@ufpe.br) on 2016-08-30T17:34:37Z No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) Janiel - Dissertação.pdf: 4742603 bytes, checksum: bfac90cf32cb35cdd6dd087a3a4bae21 (MD5) / Made available in DSpace on 2016-08-30T17:34:37Z (GMT). No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) Janiel - Dissertação.pdf: 4742603 bytes, checksum: bfac90cf32cb35cdd6dd087a3a4bae21 (MD5) Previous issue date: 2015-08-25 / FACEPE / Um novo conceito de interação tem se estabelecido pelas desenvolvedoras de jogos digitais, que adotam o uso dos gestos sendo o intermediário entre o jogador e o jogo. O presente trabalho tem como objetivo um estudo detalhado sobre quais são os gestos mais utilizados nos jogos para dispositivos móveis, analisados categoricamente por gêneros. Para execução do presente estudo, foi utilizado o método GOP (Game Ontology Project), no intuito de identificar as principais tendências nos jogos analisados e suas relações. A proposta é criar um mapa de gestos de comandos e ações completo o suficiente para indicar os gestos mais adequados a serem utilizados em um novo jogo. Os resultados possibilitaram a identificação dos tipos de gestos mais comuns nos jogos para dispositivos móveis, abrindo a oportunidade de construção de um mapeamento de gestos em situação de jogo, podendo ser útil às desenvolvedoras como uma ferramenta de orientação e de criação de jogos. / A new concept of interaction has been established by digital game developers, which adopt the use of gestures as interface between the player and the game. This work aims a detailed study of the most used gestures in games for mobile devices, categorically analyzed by gender. In this study, we used the method GOP (Game Ontology Project) in order to identify the main trends in the analyzed games and their relationships. The proposal is to create a map between command gestures and actions, complete enough to indicate the most appropriate gestures to be used in a new game. The results allowed the identification of the most common types of gestures in games for mobile devices, opening the opportunity to build a gesture mapping in game situations, which can be useful to developers as an orientation tool for creating games.

Orientadores: Ivone Panhoca, Maria de Lurdes Zanolli / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Ciencias Medicas / Made available in DSpace on 2018-08-07T03:32:00Z (GMT). No. of bitstreams: 1 Zia_Juliana_M.pdf: 453743 bytes, checksum: a3f9df2fe00b38841f34460513f326d4 (MD5) Previous issue date: 2006 / Resumo: Linguagem, aqui, é a esfera simbólica que "sustenta" e torna possível o exercício da vida em sociedade, possibilitando a estruturação das relações sociais, desta forma, a linguagem é o que faz dos indivíduos ¿seres sociais¿. E o gesto, nesta concepção teórica, é compreendido como uma esfera simbólica, como unidade sígnica, que participa das relações sócio-interativas humanas, fazendo parte do processo maior de construção do conhecimento. O objetivo geral deste trabalho foi analisar a gestualidade de uma criança com atraso de linguagem nas relações interativo-lingüísticas no contexto da terapia fonoaudiológica em grupo. Trata-se de um estudo de caso de observação participante, realizado por meio do método qualitativo, utilizando-se como diretriz teórico-metodológica os pressupostos da abordagem histórico-cultural. O estudo enfocou uma das cinco crianças que formavam um grupo terapêutico fonoaudiológico, todos do sexo masculino, com hipótese diagnóstica de atraso de linguagem, na faixa etária de 5 a 8 anos de idade. As cinco crianças foram trabalhadas em grupo durante oito meses, em sessões semanais de uma hora e meia de duração, totalizando 23 sessões terapêuticas. O material coletado foi composto de aproximadamente 34 horas de vídeo-tapes e 500 páginas de transcrições, além dos relatórios semanais das sessões. A análise de dados foi realizada por meio da análise microgenética. Este tipo de interpretação de dados é realizado por meio da análise detalhada dos recortes que refletem ações, fatos e processos dos episódios interativos ocorridos em um determinado contexto, representados aqui, na terapia fonoaudiológica em grupo. O objeto de estudo eleito foi o desenvolvimento da língua(gem), entendido como um processo dinâmico na relação do sujeito com ela, ou seja, a ¿língua em atividade¿. Esta forma de tratar os dados permite uma análise mais completa e detalhada dos processos de desenvolvimento. Adotou-se, portanto, uma posição desenvolvimental. Através da análise qualitativa dos dados, observou-se o que a interpretação e atribuição de sentidos aos gestos produzidos pela criança, por parte dos outros membros do grupo, viabilizaram sua inserção no grupo terapêutico, possibilitando que ela fosse interpretada e conseguisse interpretar, constituindo-se, assim, como sujeito daquele grupo. Concluiu-se que o fato de o gesto ser acolhido / significado naquele determinado contexto lingüístico-interativo favoreceu a (re) construção da linguagem da criança e, acima de tudo, a (re) construção da sua identidade / Abstract: Language, in this case, is the symbolic sphere that ¿sustains¿ and makes possible the exercise of life in society, providing structure in social relationships. Therefore, language is what makes individuals ¿social beings¿. And gesture, in this theoretical conception, is comprehended as a symbolic sphere, as a signal unit, that participates in human social-interactive relationships, as well as in the major process of knowledge building. The general objective of this work was to analyze gestures of a child with language delay in interactive-linguistic relationships under the context of group phonoaudiologicall therapy. This is a case study of participant observation, performed by means of the qualitative method, utilizing presuppositions as the theoretical-methodological guideline of the historical-cultural approach. The study focused on one of the five children who completed a phonoaudiological therapeutic group, all of them being males, with diagnostic hypothesis of language delay, in the age group between 5 and 8 years old. The five children were studied in group during eight months, in week sessions of one and a half hours of duration, totalizing 23 therapeutic sessions. The collected material was compounded of nearly 34 hours of videotapes and 500 pages of transcriptions, in addition to week reports of the sessions. The data analysis was performed by means of the microgenetic analysis. This kind of data interpretation is accomplished through the detailed analysis of cuttings that reflects actions, facts and processes of interactive episodes occurred in a certain context, represented in this case by the group phonoaudiological therapy. The selected object of study was the development of the language, understood as a dynamic process in its relationship with the subject, i.e. the ¿language in activity¿. This form of treating the data allows a more whole and detailed analysis of development processes. Therefore, a developmental position was adopted. Through the data qualitative analysis, it was observed that the interpretation and attribution of feelings to gestures produced by the child, through other members of the group, viabilized their insertion in the therapeutic group, allowing that they were interpreted and able to interpret, constituting this way as the subject of that group. The conclusion is that the fact of the gesture being received / signified in that determined linguistic-interactive context favored the (re) building of the child¿s language and, above all, the (re) building of their identity / Mestrado / Saude da Criança e do Adolescente / Mestre em Saude da Criança e do Adolescente

O presente estudo aborda o papel da gesticulação espontânea no ensino-aprendizagem de língua estrangeira. Verificamos em experimento o impacto da gesticulação do narrador na compreensão de narrativas por ouvintes aprendizes de língua estrangeira. Controlamos a presença e ausência do fator gestual na apresentação das narrativas. A presença de gestos aumentou o desempenho dos voluntários na tarefa de compreensão. Para entendermos melhor a relação entre gesticulação e compreensão de narrativas, correlacionamos dados de desempenho dos voluntários com medidas comportamentais de movimentação e fixação oculares registradas durante a realização dos experimentos por equipamento de rastreamento ocular / The present study investigates the role of spontaneous gesticulation in the process of teaching and learning a foreign language. We studied experimentally the impact of the narrators gesticulation on the comprehension of narratives by students of English as a foreign language. We controlled the presence and absence of hand gesticulation in the presentation of the stimuli. The presence of gestures increased volunteers performance in the comprehension task. For a better understanding of the relation between gesticulation and comprehension of narratives, we correlated data from volunteers perfomance with behavioral measures of eye movement and fixation, captured during the experiment by eyetracking methodology

Utiliser une interface visuo-tactile peut être une gêne lorsqu'il est nécessaire de rester mobile et conscient de son environnement. Cela s'avère particulièrement problématique en milieu hostile comme pour la commande d'un drone militaire de contact.Dans ces travaux nous faisons l'hypothèse que le geste est une modalité de commande moins contraignante puisqu'elle n'impose pas de visualiser ni de manipuler une interface physique.Ainsi, différents travaux nous ont permis de confirmer d'une part, les avantages pratiques, et d'autre part, la faisabilité technique de la commande d'un robot mobile par geste.Tout d'abord, l'étude théorique du geste a permis de construire un modèle d'interaction. Celui-ci consiste en l'activation de commandes automatiques par la réalisation de gestes sémaphoriques normalisés. Des messages sonores permettent de renseigner l'opérateur et un mécanisme de confirmation sécurise l'interaction.Ensuite, le dictionnaire des gestes à utiliser a été constitué. Pour cela, une méthodologie a été proposée et appliquée : des utilisateurs proposent puis élisent les gestes les plus pertinents.Notre modèle d'interaction et le vocabulaire gestuel ont ensuite été évalués. Une étude en laboratoire nous a permis de montrer que l'interaction gestuelle telle que proposée est simple à apprendre et utiliser et qu'elle permet de conserver une bonne conscience de l'environnement.Un système interactif complet a ensuite été développé. Son architecture a été déduite du modèle d'interaction et une brique de reconnaissance gestuelle a été mise en oeuvre. En marge des méthodes classiques, la brique proposée utilise un principe de description formelle des gestes avec une grammaire régulière.Finalement, ce système a été évalué lors de tests utilisateurs. L'évaluation par des participants militaires a confirmé notre hypothèse de la pertinence du geste pour une interaction visuellement et physiquement moins contraignante. / Using a visuo-tactil interface may be restrictive when mobility and situation awareness are required. This is particularly problematic in hostile environment as commanding a drone on a battlefield.In the work presented here, we hypothesize that gesture is a less restrictive modaility as it doesn't require to manipulate nor to look at a device.Thus we followed a user-centered approach to confirm practical advantages and technical feasibility of gestural interaction for drones.First, the theoretical study of gestures allowed us to design an interaction model. It consists on activating commands by executing standardized semaphoric gestures. Sound messages inform the user and a confirmation mechanism secure the interaction.Second, a gestural vocabulary has been built. To do so, a methodology has been proposed and used : end users elicited then elected the most appropriate gestures.Then, the interaction model and the selected gestures have been evaluated. A laboratory study showed that they are both easy to learn and use and helps situation awareness.An interactive system as then been developed. It's architecture has been deducted from our interaction model and a gesture recognizer as been buit. Different from usual methods, the recognizer we proposed is based on formal description of gestures using regular expressions.Finaly, we conducted a user testing of the proposed system. The evaluation by end-users confirmed our hypothesis that gestures are a modality less distractive both visualy and physicaly.

Conducting gestures and facial expressions can be interpreted with wide variance by musicians, even within ensembles with a close range of technical mastery and experience. In this study, I examined the interpretations of a music conductor’s nonverbal communication to collegiate wind ensemble students and the accompanying pedagogical considerations when leading live performers. The conceptual framework of the study was kinesics, “the study of body movements, facial expressions, and gestures” (Ottenheimer, 2009, p. 160), and more specifically, Ekman and Friesen’s (1969) categories of nonverbal communication. Within this framework, the two categories I used specifically were emblems- nonverbal signals from the body representing a verbal message, and affect displays- characterizations of an emotion or other message depicted primarily on the face. Utilizing gesture descriptions compiled by Sousa (1988), I created a video stimulus to interview students on their reactions to 21 gestures of the hands, arms, and torso, as well as 10 naturally occurring facial expressions while conducting. Using the conducting video as the stimulus, I interviewed 80 college students at nine college campuses. Students participated in an individual 30-minute interview where they watched each of the 31 video excerpts and gave verbal feedback about what they perceived as the message of each of the gestures or facial expressions. Data were analyzed and compared to Sousa’s (1988) descriptions of each gesture from which the conductor attempted to demonstrate on the video. Utilizing Ekman and Friesen’s (1969) metric of 70% recognition to code a response as an emblem, 16 of the 21 gestures (76%) were discovered to be musical emblems, compared to 71% in Sousa’s (1988) study. Only 12 out of 21 gestures were identified as emblems in both studies (57%). Categories of the strongest prevalence in the current study of emblems included dynamics and tempo changes. Results from the 10 videos of facial expression netted more than ten different themes per affect display, each with diverse descriptions of musical and emotional messages. Overall results showed the small muscle movements of the face are capable of multi-message and multi-signal semiotic functions (Ekman & Friesen, 1978) with robust descriptions that can change rapidly in significant ways.

Effective collaboration in a team is a crucial skill. When people interact together to perform physical tasks, they rely on gestures to convey instructions. This thesis explores gestures as means to assess physical collaborative task understanding. This research proposes a framework to represent, compare, and assess gestures’ morphology, semantics, and pragmatics, as opposed to traditional approaches that rely mostly on the gestures’ physical appearance. By leveraging this framework, functionally equivalent gestures can be identified and compared. In addition, a metric to assess the quality of assimilation of physical instructions is computed from gesture matchings, which acts as a proxy metric for task understanding based on gestural analysis. The correlations between this proposed metric and three other task understanding proxy metrics were obtained. Our framework was evaluated through three user studies in which participants completed shared tasks remotely: block assembly, origami, and ultrasound training. The results indicate that the proposed metric acts as a good estimator for task understanding. Moreover, this metric provides task understanding insights in scenarios where other proxy metrics show inconsistencies. Thereby, the approach presented in this research acts as a first step towards assessing task understanding in physical collaborative scenarios through the analysis of gestures.

Domestic IoT appliances like smart speakers, smart locks and robot vacuum cleaners are usually connected through smartphone apps to provide additional functionality and remote control. Although smartphones have many different sensors and actuators, these apps provide a primarily graphical user interface with these appliances. To explore a more somatically engaging experience the prototype JoyTilt was designed, developed and tested with users. JoyTilt enables its user to control a robot vacuum cleaner like a toy car by tilting their phone in the direction they want it to go. This study found that the participants had their gaze focused on the robotic vacuum cleaner while controlling it and that the participants had both positive and negative bodily experiences with JoyTilt. Interviews with the participants provide suggestions for balancing control of robot vacuum cleaners and keeping the robot’s autonomy using JoyTilt. In this study the aesthetics of the material, choice of materials and choice of interaction model come together in the design of JoyTilt to shape the human-robot relationship. Lastly, the thesis highlights the values of further considering the bodily experience when designing apps. / Uppkopplade produkter för hemmabruk så som smarta högtalare, smarta lås och larm eller robotdammsugare kopplas till våra smartphones via appar som möjliggör ytterligare funktioner, exempelvis fjärrstyrning och schemaläggning. Trots att smartphones innehåller en mängd olika sensorer används i regel primärt grafiskt gränssnitt i dessa appar. Genom att designa, utveckla och testa prototypen JoyTilt utforskas en somaestetisk användarupplevelse med fokus på den fysiska upplevelsen. Med appen JoyTilt kan en robotdammsugare styras likt en radiostyrd bil genom att luta smartphonen. I testerna med JoyTilt hade deltagarna sin blick fästa på robotdammsugaren när de styrde den. Deltagarna hade både positiva och negativa kroppsliga upplevelser av Joytilt. Från intervjuerna med deltagarna kom förslag på sätt att balansera kontroll av robotdammsugaren med dess autonomi med hjälp av JoyTilt. Val av designmaterial, dess estetik och interaktionsmodell bidrog till utformningen av JoyTilt som i sin tur påverkade relationen mellan människa och robot. Slutligen understryker den här studien värdet av att ta den kroppsliga upplevelsen i beaktning och nyttja det ofta förbisedda fysiska designutrymmet i appdesign.

Freehand gestures have long been considered to potentially deliver natural, intuitive, terse but powerful human-computer interaction techniques. Over years, researchers have been attempting to employ freehand gestures as an alternative input modality to the conventional devices (e.g. keyboard and mouse) in a wide array of application domains, and a huge number of gesture recognition systems and gesture-based interaction techniques have been created in lab. However, a fundamental question remains: is it possible to establish an interaction framework so that we may approach freehand gestural interaction from a systematic perspective, and design coherent and consistent freehand gesture-based human-computer interaction experience? Existing research tends to focus on the technologies that enable the gestural interaction, or on the novel design of gestural interaction techniques for specific tasks and applications. Such "point designs" are claimed to be insufficient, and an existing application-specific design lends very limited insights and guidance to design problems in another application. An interaction framework allows us to move from individual designs to a more holistic approach. The goal of this research is to construct a framework to support a systematic approach for designing freehand gesture-based interactions. Toward this goal our research began with a review and examination of the gesture interaction literature, followed by an analysis of the essential components of an interaction framework. We then proposed and justified the scope of research and the approach we took to construct the interaction framework. We have designed and evaluated (analytically and empirically) gestural interaction techniques for two broad categories of freehand gestures we specified — spatial gestures, and surface gestures. In the design activity, we have discovered and proposed the core design principles and guidelines, and validated them via user studies. Finally, we assessed the ability of the freehand gesture interaction framework we have constructed to help designers create new applications and designs, by putting together a few proof-of-concept examples of a coherent and consistent freehand gesture user interface. / Ph. D.

Notre projet de recherche ambitionne de mesurer l’impact sur les activités d’apprentissage des élèves du recours, par les acteurs de la situation didactique, à un enchainement de canaux de communication. Ce phénomène d’enchaînement orchestrant les activités du faire apprendre et de l’apprendre, sont nommés transitions codiques. Il s’agit dans un premier temps d’en montrer l’enjeu scientifique dans le champ de la didactique. Ensuite, il est question d’en circonscrire les caractéristiques essentielles. Enfin, il s’agit d’étudier l’impact différentiel de ces caractéristiques sur les gestes d’étude des élèves. Cette étude, en s’inscrivant dans le paradigme de l’interactionnisme symbolique, exige l’exploitation croisée de trois cadres théoriques en didactique : celui de la théorie des situations didactiques (TSD) qui identifie des typologies de situations organisées autour d’étapes dans la construction d’une connaissance en jeu, celui de la théorie anthropologique du didactique (TAD) qui articule au savoir en jeu la notion de rapport au savoir, et qui substitue cette notion de savoir à celle de praxéologie, et enfin celui de la théorie de l’action conjointe en didactique (TACD), organisé autour de l’idée essentielle que le savoir scolaire ne se définit pas de façon descendante et prescriptive, mais relève aussi d’une transposition ascendante qui induit la nécessaire prise en compte des dimensions d’épistémologie pratique de l’enseignant dans son institution, des actions didactiques et de ces différents descripteurs. Le dispositif mis en place, nous inscrit dans les perspectives comparatistes en didactique selon une approche descriptive et compréhensive en croisant des entretiens, des observations vidéos, et traces prises au vol.Les résultats obtenus mettent en évidence que les transitions codiques constituent l’essentiel du temps d’interaction et s'organisent selon une charge directionnelle, des délais, des degrés d’ostension qui évoluent dans le temps et qui renseignent sur le registre d'action enseignante. Ces caractéristiques varient également selon le mode de convocation des savoirs qui les sous-tendent, à l’interface entre cognition consciente et intuition. Elles structurent une rythmique de l’intervention et de l’interaction différentielle au regard des niveaux de pratique et du genre. Elles pilotent en conséquence les gestes d’étude des élèves. / Our research project aims to measure the impact on pupils' learning activities of the use, by the actors of the didactic situation, of a series of channels of communication. This phenomenon of sequencing orchestrating the activities of learning and learning are called code transitions. It is a question of first showing the scientific stake in the field of didactics. Then, it is a question of circumscribing its essential characteristics. Finally, it is a question of studying the differential impact of these characteristics on the students' gestures of study.This study, in keeping with the paradigm of symbolic interactionism, requires the cross-exploitation of three theoretical frameworks in didactics: that of the theory of didactical situations (TSD) which identifies typologies of situations organized around stages in the construction of a knowledge at stake, that of the anthropological theory of didactics (TAD) which articulates to the knowledge at stake the notion of relation to knowledge, and which substitutes this notion of knowledge for that of praxeology, and finally that of the theory of joint action in didactics (TACD), organized around the essential idea that school knowledge is not defined in a top-down and prescriptive way, but is also an upward transposition which induces the necessary consideration of the dimensions of practical epistemology of the teacher in his institution, didactic actions and these different descriptors.The system put in place, inscribes us in the comparatist perspectives in didactics according to a descriptive and comprehensive approach by crossing interviews, video observations, and traces taken in flight.The results show that the codic transitions constitute the bulk of the interaction time and are organized according to a directional charge, delays, degrees of evolution which evolve over time and which inform the action register. teacher. These characteristics also vary according to the mode of convocation of the knowledge that underlies them, at the interface between conscious cognition and intuition. They structure a rhythm of intervention and differential interaction in terms of levels of practice and gender. They pilot the pupils' gestures of study accordingly