Dans l'optique d'une organologie de la performance, on commence par élaborer, autour de la notion de système haptique d'un instrument de musique, un cadre taxinomique et une typologie aboutissant à un questionnement sur l'identité conceptuelle d'un instrument-objet. Une approche inductive sur le cas exemplaire de la guitare permet alors de définir une distance potentielle dans l'interaction instrument/instrumentiste ; puis, par modélisation d'extraits de pièces musicales, de considérer l'optimisation gestique comme un argument de distanciation qui consiste en une abstraction dans le champ logique de l'espace haptique (géométrisation). Pour finir, on rend compte de l'aperception du musical par l'instrumentiste, dans le cadre d'une exécution et au travers de son instrument, par un modèle dichotomique où les concepts de processus de solmisation et de vectorisation s'avèrent proportionnels aux aspects stylistiques des œuvres et de leurs interprétations. / From the perspective of an organology of performance, we first developed, around the concept of haptic system for a musical instrument, a taxonomic framework and typology leading to a questioning about the conceptual identity of an instrument-object. An inductive approach to the exemplary case of the guitar allows us to define a potentially distance in the interaction instrument/instrumentalist ; then, by modeling excerpts of musical works, to consider gestural optimization as an argument of distancing which is an abstraction in the logical field of haptic space (geometric). Finelly, we report the apperception of musical by the instrumentalist, as part of a performance and through his instrument, with a dichotomic model where the concepts of solmisation process and vectorization are proportional to the stylistic aspects of the works and their interpretations.

Com o constante crescimento tecnológico, é bastante comum deparar-se com um display público em lugares de grande concentração de pessoas, como aeroportos e cinemas. Apesar de possuírem informações úteis, esses displays poderiam ser melhor aproveitados se fossem interativos. Baseando-se em pesquisas sobre a interação com displays grandes e as características próprias de um display colocado em um espaço público, busca-se uma maneira de interação que seja adequada a esse tipo de situação. O presente trabalho introduz um método de interação por gestos sem necessitar que o usuário interagente segure ou tenha nele acoplado qualquer dispositivo ao interagir com um display público. Para realizar as tarefas que deseja, o usuário só precisa posicionar-se frente ao display e interagir com as informações na tela com suas mãos. São suportados gestos para navegação, seleção e manipulação de objetos, bem como para transladar a tela de visualização e ampliá-la ou diminui-la. O sistema proposto é construído de forma que possa funcionar em aplicações diferentes sem um grande custo de implantação. Para isso, é utilizado um sistema do tipo cliente-servidor que integra a aplicação que contém as informações de interesse do usuário e a que interpreta os seus gestos. É utilizado o Microsoft Kinect para a leitura dos movimentos do usuário e um pós-processamento de imagens é realizado de modo a detectar se as mãos do usuário se encontram abertas os fechadas. Após, essa informação é interpretada por uma máquina de estados que identifica o que o usuário está querendo executar na aplicação cliente. Afim de avaliar o quão robusto o sistema se portaria em um ambiente público real, são avaliados critérios que poderiam interferir na tarefa interativa, como a diferença de luminosidade do ambiente e a presença de mais pessoas no mesmo local de interação. Foram desenvolvidas três aplicações a título de estudo de caso e cada uma delas foi avaliada de forma diferente, sendo uma delas utiliza para fins de avaliação formal com usuários. Demonstrados os resultados da avaliação realizada, conclui-se que o sistema, apesar de não se portar corretamente em todas situações, tem potencial de uso desde que sejam contornadas suas deficiências, a maior parte das quais originária das próprias limitações inerentes ao Kinect. O sistema proposto funciona suficientemente bem para seleção e manipulação de objetos grandes e para aplicações baseadas em interação do tipo pan & zoom, como navegação em mapas, por exemplo, e não é influenciado por diferenças de iluminação ou presença de outras pessoas no ambiente. / With the constant technological growth, it is quite common to come across a public display in places with high concentration of people, such as airports and theaters. Although they provide useful information, these displays could be better employed if they were interactive. Based on research on topics of interaction with large displays and the characteristics of a display placed in a public space, a way of interaction that is suitable for this kind of situation is searched. This paper introduces a method of interaction by gestures without requiring that the interacting user take hold or have to him attached any device to interact with a public display. To accomplish the tasks he wants, he needs just to position himself in front of the display and to interact with the information on the screen with his hands. Gestures supported provide navigation, selection and manipulation of objects as well as to pan and zoom at the screen. The proposed system is constructed so that it works in different applications without a large installation cost. In order to achieve this, the system implements a client-server model application that is able to integrate the part that contains the useful information to the user and the one that interprets his gestures. The Microsoft Kinect is used for reading the user’s movements and techniques of image processing are performed to detect if the user’s hands are open or closed. After this information is obtained, it runs through a state machine that identifies what the user is trying to do in the application. In order to evaluate how robust the system is in a real public environment, some criteria that could interfere with the interactive task are evaluated, as the difference in brightness in the environment and the presence of another people in the same place of interaction. Three applications were developed as a case study and each one was evaluated differently, one of them being used for formal user evaluation. Given the results of the performed tasks, it is possible to conclude that the system, although not behaving correctly in all situations, has potential use if its difficulties are circumvented, most of which come from Kinect’s own inherent limitations. The proposed system works well enough for selection and manipulation of large objects and for use in applications based on pan & zoom, like those that supports map navigation, for example, and difference of ilumination or the presence of other persons on the environment does not interfere with the interaction process.

Esse trabalho partiu da reflexão sobre interfaces gráficas sofisticadas e pouco intuitivas e o impacto das mesmas na modelagem tridimensional. A partir dessa reflexão, a pesquisa foi direcionada para o estudo de interfaces gestuais. Adotou-se, como foco do estudo, a técnica de geração de modelos tridimensionais, chamada origami por seu caráter exploratório, sistemático e sua aproximação com a área da matemática, engenharia, arquitetura e design. O problema de pesquisa consistia em identificar padrões de gestos da mão e diretrizes para uma interface gestual voltada à construção de modelos tridimensionais, a partir das técnicas de origami. A pesquisa buscou fundamentação na bibliografia e na análise sistemática de vídeos de modelagem física referentes aos movimentos da mão e ao comportamento dos modelos. Com os dados levantados foram gerados esquema e gráficos para análise. Os dados gerados auxiliaram na seleção de um vocabulário de gestos e de diretrizes para interface e comportamento dos modelos no ambiente digital que passaram por testes de aplicabilidade no ambiente físico. Os resultados gerados visaram direcionar para desenvolvedores de softwares sobre as principais características da modelagem de origami no meio digital com utilização de interface gestual. / This work was based on the consideration of sophisticated and somewhat intuitive graphical interfaces and their impact on three-dimensional modeling. Based on this discussion, the research is directed towards the gestural interfaces studies. It was adopted, as the focus of study, the threedimensional models generating technique, called origami. Because of his approach to the mathematics, engineering, architecture and design field and his exploratory and systematic characteristic. The research problem was to identify gestures hand patterns and guidelines for a gestural interface focused on building three-dimensional models, from the origami techniques. The work sought, justification in the literature and systematic analysis of physical modeling video related to hand movements and models behavior. With the collected data, graphics and analysis scheme were generated. The generated data helped in the selection of a gestures vocabulary and guidelines for interface and behavior of the models in the digital environment. Those were submitted on applicability tests in the physical environment. The results generated, aims give some directions for developers of software about the main features of the origami modeling in digital media using gestural interface.

In this thesis, we address the question of gesture interaction on mobile device. These devices, now common, differ from conventional computers primarily by the input devices the user interact with (screen size small but tactile, various sensors such as accelerometers) as well as the context in which they are used. The work presented here is an exploration of the vast area of interaction techniques on these mobile devices. First we try to structure this space by focusing on the techniques based on accelerometers for which we propose a taxonomy. Its descriptive and discriminant power is validated by and the classification of thirty-seven interaction techniques in the literature. Second we focus on the achievement of gestural interaction techniques for these mobile devices. With TouchOver, we show that it is possible to take advantage of complementary two-channels input (touch screen and accelerometers) to add a state to the finger-drag, thus enriching the interaction. Finally, we focus on mobile device menus and offer a new form of sign language menus. We discuss their implementation with the GeLATI software library that allows their integration into a pre-existing GUI toolkit.

An important element in understanding the evolutionary origin of human language is to explore homologous traits in cognition and communication between primates and humans (Burling, 1993, Hewes, 1973). One proposed modality of language evolution is that of gestural communication, defined as communicative movements of hands without using or touching objects (de Waal, 2003). While homologies between primate calls and language have been relatively well explored, we still have a limited understanding of how cognitive abilities may have shaped the characteristics of primate gestures (Corballis, 2003). Chimpanzees (Pan troglodytes) are our closest living relatives and display some complex cognitive skills in various aspects of their gestural behaviour in captivity (de Waal, 2003, Pollick and de Waal, 2007). However, it is not yet currently clear to what extent these abilities seen in captive apes are typical of chimpanzees in general and to what extent cognitive capacities observed in captive chimpanzees have been enhanced by the socio-cultural environment of captivity such as language training. In this Ph.D. research, I investigated the cognitive skills underlying gestural communication in both wild and language trained chimpanzees, with a special focus on the repertoire and the intentionality of production and comprehension. The study of cognitive skills underlying the production of the repertoire and the role of intentionality is important because these skills are cognitively demanding and are a prerequisite in human infants for their ability to acquire language (Baldwin, 1995, Olson, 1993). My research suggests that chimpanzee gestural communication is cognitively complex and may be homologous with the cognitive skills evident in pre-verbal infants on the cusp of language acquisition. Chimpanzees display a multifaceted and complex signal repertoire of manual gestures. These gestures are the prototypes, within which there is variation, and between which the boundaries are not clear-cut, but there is gradation apparent along several morphological components. Both wild and language trained chimpanzees communicate intentionally about their perceived desires and the actions that they want the recipients to undertake. They do not just express their emotions, but they communicate flexibly by adjusting their communicative tactics in response to the comprehension states of the recipient. Whilst chimpanzees communicate their intentions flexibly, the messages conveyed are specific. However, recipients comprehend gestures flexibly in light of the signaller’s overall intentions. Whilst wild and language trained chimpanzee gestural communication revealed similar cognitive characteristics, language trained chimpanzees outperformed wild apes in that they had ability to use signals which made distinctions that human deictic words can make. Whilst these differences between wild and language trained chimpanzees may be due to the different methodological approaches used, it is conceivable that language training may have influenced captive ape cognitive skills in the representational domain. These results from wild and language trained chimpanzees indicate that chimpanzees possess some form of cognitive skills necessary for language development and that cognitive skills underlying repertoire and use in chimpanzees are a shared capacity between humans, other apes and a common ancestor. These findings render theories of the gestural origins of language more plausible. Related publications: 1. Roberts, A. I., Vick, S.-J., Roberts, S. G. B., Buchanan-Smith, H. M. & Zuberbühler, K. 2012. A structure-based repertoire of manual gestures in wild chimpanzees: Statistical analyses of a graded communication system. Evolution and Human Behavior, Published online: http://dx.doi.org/ 10.1016/j.evolhumbehav.2012.05.006 2. Roberts, A. I., Vick, S.-J. & Buchanan-Smith, H. 2012. Usage and comprehension of manual gestures in wild chimpanzees. Animal Behaviour, Published online: http://dx.doi.org/10.1016/j.anbehav.2012.05.022

This portfolio of compositions is a logbook on how trans-disciplinary collaboration, gestural embodiment of sound and social context influenced my work as a composer between 2008 and 2013. In most pieces, community based environments with experts in fields other than music were crucial to explore new sound worlds and creative processes beyond my solely scope. While music composition is traditionally a lonely act, this collaborative approach allowed me to repeatedly answer two questions that fascinate me: How can other fields of knowledge inform music? And: How can music inform other fields of knowledge? In some cases, I even used the score to foster transdisciplinary collaboration, like in "Nadir" and "Bisbiglio Qualcosa en el mio Orecchio" where collaboration between musicians and non-musicians is necessary. / Music

The main purpose of this research is to discuss the specificity of the microphone as a gestural tool through which the sequence organization of media two-party interviews is accomplished. The study focuses on the practical communicative problems of microphone operations in a media setting where the parties have alternating turns, and addresses the question of who of the participants speaks next and for how long. It is particularly concerned with investigating what the participating parties can do with the microphone, what it accomplishes, and how it is used as a tool for interaction with an audience. We particularly focus on how microphone moves can make use see how how people orient to emergent content structures in talk, and how microphone performs as a device for confirmation of verbal turns.

Tyrimas atliktas Kauno apskrities kurčiųjų ir neprigirdinčiųjų ugdymo centre. Analizuota mokslinė literatūra, Centro dokumentai, anketinio klausimyno metodu apklausti šioje institucijoje dirbantys pedagogai. Atlikus tyrimą paaiškėjo, kad kitaip organizuojant patį mokymo procesą, patobulinus kai kuriuos jo elementus, mokymo procesas vyktų efektyviau. Mokymo proceso metu mokytojas susiduria su tokiomis problemomis: nepritaikyti vadovėliai; vaikai nesupranta daugelio, kartais net ir labai elementarių, aiškinamų dalykų; nesunorminta gestų kalba ir nepakankamas jos mokėjimas bei kiti sunkumai perteikiant medžiagą gestų kalba, kurie apsunkina mokymo procesą. Neigiamą įtaka taip pat daro ir tai, kad vaikai dažnai nesidomi dalyku – tuomet išlaikyti jų dėmesį ir perteikti mokymo turinį yra. Tyrimas taip pat atskleidė, kad nedrausmingi moksleiviai trukdo mokymo procesui. Nepakankamas dėmesys skiriamas plėtoti bilingvinio metodo plėtoti taikymui, siekiant, kad jis įsitvirtintų mokymo procese kaip pagrindinis, nes šitaip perteikiamas mokymo turinys yra priimtiniausias kartu mokant kurčius ir neprigirdinčius moksleivius. Norint išvengti, ar bent sumažinti šias problemas, reikėtų kuo daugiau rengti kurtiesiems ir neprigirdintiesiems pritaikytų mokymo priemonių, ypač vadovėlių. Reikėtų atsižvelgti ir į tai, kad šie moksleiviai, kaip iš prigimties bilingvistai, būtų mokomi bilingviniu metodu, kuris jiems yra tinkamiausias. Kad tai būtų galima padaryti, reikia kuo skubiau vystyti pedagogų... [toliau žr. visą tekstą] / The research was accomplished at Kaunas County Deaf and Hearing Impaired Educational Centre. Scientific literature and Centre documentation were analysed, educators working at the Centre were interviewed in method of questionnaire survey for its purpose. The research revealed that, the process of education was more successful, if the process of teaching itself was differently arranged, and some of its elements were improved. In the course of education, teachers encounter different problems: inapplicable texbooks, students unable to perceive most of, even elementary, topics, non-standardization of gestural language, or inaptitude to speak it appropriatelly, as well as other difficulties in conveying material in gestural language, aggrevate the teaching. The negative signs also occure as students often are not interested in the subject taught – it becomes especially complicated to attract and retain their focus of attention, and subsequently to convey knowledge. The research revealed, that undisciplined students intervene process of teaching. More attention should be paid to develope bilingual method, and seek to introduce it as the main in the process of teaching, as this pattern is the most acceptable to communicate knowledge to boths deaf and hearing impaired students. On purpose to avoid or at least to decrease these problems, more learning facilities, especially handbooks, for deaf or hearing impaired should be prepared. Considering these students being inherently... [to full text]

To investigate the ability for animals to understand human communication signals and the communication between animals and humans, scientists often investigate the understanding of human gestural cues. Dogs (Canis lupus familiaris) which have a long history of co-evolution with humans have been shown to make good use of human gestural cues. In the present study I investigated whether dogs in general understand a human pointing gesture and if there are differences between sex, age or breeds. In total 46 dogs of different breeds participated in the study. The study was carried out in a dog center in Linköping, Hundens och djurens beteendecenter. To test if dogs understand human pointing gestures, a two-way object choice test were used, where an experimenter pointed at a baited bowl at a distance of three meter from the dog. The results showed that dogs in general can understand human pointing gestures. However, no significant differences were found for sex, age or breeds. As a conclusion, I found that dogs in general can understand human pointing gestures, but sex, age or breed did not affect the ability.

Com o constante crescimento tecnológico, é bastante comum deparar-se com um display público em lugares de grande concentração de pessoas, como aeroportos e cinemas. Apesar de possuírem informações úteis, esses displays poderiam ser melhor aproveitados se fossem interativos. Baseando-se em pesquisas sobre a interação com displays grandes e as características próprias de um display colocado em um espaço público, busca-se uma maneira de interação que seja adequada a esse tipo de situação. O presente trabalho introduz um método de interação por gestos sem necessitar que o usuário interagente segure ou tenha nele acoplado qualquer dispositivo ao interagir com um display público. Para realizar as tarefas que deseja, o usuário só precisa posicionar-se frente ao display e interagir com as informações na tela com suas mãos. São suportados gestos para navegação, seleção e manipulação de objetos, bem como para transladar a tela de visualização e ampliá-la ou diminui-la. O sistema proposto é construído de forma que possa funcionar em aplicações diferentes sem um grande custo de implantação. Para isso, é utilizado um sistema do tipo cliente-servidor que integra a aplicação que contém as informações de interesse do usuário e a que interpreta os seus gestos. É utilizado o Microsoft Kinect para a leitura dos movimentos do usuário e um pós-processamento de imagens é realizado de modo a detectar se as mãos do usuário se encontram abertas os fechadas. Após, essa informação é interpretada por uma máquina de estados que identifica o que o usuário está querendo executar na aplicação cliente. Afim de avaliar o quão robusto o sistema se portaria em um ambiente público real, são avaliados critérios que poderiam interferir na tarefa interativa, como a diferença de luminosidade do ambiente e a presença de mais pessoas no mesmo local de interação. Foram desenvolvidas três aplicações a título de estudo de caso e cada uma delas foi avaliada de forma diferente, sendo uma delas utiliza para fins de avaliação formal com usuários. Demonstrados os resultados da avaliação realizada, conclui-se que o sistema, apesar de não se portar corretamente em todas situações, tem potencial de uso desde que sejam contornadas suas deficiências, a maior parte das quais originária das próprias limitações inerentes ao Kinect. O sistema proposto funciona suficientemente bem para seleção e manipulação de objetos grandes e para aplicações baseadas em interação do tipo pan & zoom, como navegação em mapas, por exemplo, e não é influenciado por diferenças de iluminação ou presença de outras pessoas no ambiente. / With the constant technological growth, it is quite common to come across a public display in places with high concentration of people, such as airports and theaters. Although they provide useful information, these displays could be better employed if they were interactive. Based on research on topics of interaction with large displays and the characteristics of a display placed in a public space, a way of interaction that is suitable for this kind of situation is searched. This paper introduces a method of interaction by gestures without requiring that the interacting user take hold or have to him attached any device to interact with a public display. To accomplish the tasks he wants, he needs just to position himself in front of the display and to interact with the information on the screen with his hands. Gestures supported provide navigation, selection and manipulation of objects as well as to pan and zoom at the screen. The proposed system is constructed so that it works in different applications without a large installation cost. In order to achieve this, the system implements a client-server model application that is able to integrate the part that contains the useful information to the user and the one that interprets his gestures. The Microsoft Kinect is used for reading the user’s movements and techniques of image processing are performed to detect if the user’s hands are open or closed. After this information is obtained, it runs through a state machine that identifies what the user is trying to do in the application. In order to evaluate how robust the system is in a real public environment, some criteria that could interfere with the interactive task are evaluated, as the difference in brightness in the environment and the presence of another people in the same place of interaction. Three applications were developed as a case study and each one was evaluated differently, one of them being used for formal user evaluation. Given the results of the performed tasks, it is possible to conclude that the system, although not behaving correctly in all situations, has potential use if its difficulties are circumvented, most of which come from Kinect’s own inherent limitations. The proposed system works well enough for selection and manipulation of large objects and for use in applications based on pan & zoom, like those that supports map navigation, for example, and difference of ilumination or the presence of other persons on the environment does not interfere with the interaction process.