Hand gesture recognition is an active area of research in recent years, being used in various applications from deaf sign recognition systems to human-machine interaction applications. The gesture recognition process, in general, may be divided into two stages: the motion sensing, which extracts useful data from hand motion; and the classification process, which classifies the motion sensing data as gestures. The existing vision-based gesture recognition systems extract 2-D shape and trajectory descriptors from the visual input, and classify them using various classification techniques from maximum likelihood estimation to neural networks, finite state machines, Fuzzy Associative Memory (FAM) or Hidden Markov Models (HMMs). This thesis presents the framework of the vision-based Hand Motion Understanding (HMU) system that recognises static and dynamic Australian Sign Language (Auslan) signs by extracting and classifying 3-D hand configuration data from the visual input. The HMU system is a pioneer gesture recognition system that uses a combination of a 3-D hand tracker for motion sensing, and an adaptive fuzzy expert system for classification. The HMU 3-D hand tracker extracts 3-D hand configuration data that consists of the 21 degrees-of-freedom parameters of the hand from the visual input of a single viewpoint, with an aid of a colour coded glove. The tracker uses a model-based motion tracking algorithm that makes incremental corrections to the 3-D model parameters to re-configure the model to fit the hand posture appearing in the images through the use of a Newton style optimisation technique. Finger occlusions are handled to a certain extent by recovering the missing hand features in the images through the use of a prediction algorithm. The HMU classifier, then, recognises the sequence of 3-D hand configuration data as a sign by using an adaptive fuzzy expert system where the sign knowledge are used as inference rules. The classification is performed in two stages. Firstly, for each image, the classifier recognises Auslan basic hand postures that categorise the Auslan signs like the alphabet in English. Secondly, the sequence of Auslan basic hand postures that appear in the image sequence is analysed and recognised as a sign. Both the posture and sign recognition are performed by the same adaptive fuzzy inference engine. The HMU rule base stores 22 Auslan basic hand postures, and 22 signs. For evaluation, 44 motion sequences (2 for each of the 22 signs) are recorded. Among them, 22 randomly chosen sequences (1 for each of the 22 signs) are used for testing and the rest are used for training. The evaluation shows that before training the HMU system correctly recognised 20 out of 22 signs. After training, with the same test set, the HMU system recognised 21 signs correctly. All of the failed cases did not produce any output. The evaluation has successfully demonstrated the functionality of the combined use of a 3-D hand tracker and an adaptive fuzzy expert for a vision-based sign language recognition.

Thesis (Ph. D.)--University of Toronto, 2004. / Advisers: Rick Volpe; Deb Fels.

Thesis (M.A.)--York University, 2006. Graduate Programme in Psychology. / Typescript. Includes bibliographical references (leaves 48-55). Also available on the Internet. MODE OF ACCESS via web browser by entering the following URL: http://gateway.proquest.com/openurl?url_ver=Z39.88-2004&res_dat=xri:pqdiss&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&rft_dat=xri:pqdiss:MR19674

Thesis (Ph.D.) - University of St Andrews, August 2008.

In human-human dialogues, face-to-face meetings are often preferred over phone conversations.One explanation is that non-verbal modalities such as gesture provide additionalinformation, making communication more efficient and accurate. If so, computerprocessing of natural language could improve by attending to non-verbal modalitiesas well. We consider the problem of sentence segmentation, using hand-annotatedgesture features to improve recognition. We find that gesture features correlate wellwith sentence boundaries, but that these features improve the overall performance of alanguage-only system only marginally. This finding is in line with previous research onthis topic. We provide a regression analysis, revealing that for sentence boundarydetection, the gestural features are largely redundant with the language model andpause features. This suggests that gestural features can still be useful when speech recognition is inaccurate.

Experiência de escrita potencializada pelo gesto. Gesto desdobrado em vibrações que ferem o espírito de forma incontornável. Contorno do acumulam nas mãos, resumidos aos olhos. Uma ilha onde se força o exílio, apreende o gesto e protela o intraduzível. Diferente da língua gestual, do código de acontecimento com margens e paredes, a ferida no sótão, na praia, na montanha. Ocupar as feridas que me ferem com suas respectivas roupagens, marcar o gesto com roupa, alimento, utensílio. Refratariamente, para ambos os lados: os gestos se multiplicam ao infinito, digeridos pelas entranhas e se gesto - a agressividade da marca, do que escorre como sangue, da mão morta, da epilepsia. / Writing experience increased by the gesture. Gesture deployed into vibrations that hurt the spirit so compelling. Outline of the event with the banks and walls, the wound in the attic, on the beach, the mountains. Occupy the wounds that hurt me with outfits, to mark the gesture with clothing, food and devices. Refractory to both sides: the gestures are multiplied to infinity, digested in the bowels and accumulate in the hands, as summarized in the eyes. Islands of a forced exile, seize the gesture and delays the untranslatable. Unlike sign language, the code of gesture – the aggressiveness of the brands, that which drips like blood, the dead hand, the epilepsy.

This dissertation examines speakers’ production of speech and representational gesture. It utilizes the Repetition Effect as the investigative tool. The Repetition Effect appears to vary by the tendency for some items to shorten when repeating, at least under the condition that speakers can primarily operate by their assumption of the state of knowledge of the listener. In speech, a highly conventionalized form of performance, word duration reduces within the same stretch of coherent discourse; then, it resets in the first mention of a new stretch of coherent discourse regardless of the state of knowledge to the speaker or the listener. Therefore, the Repetition Effect in speech is best analyzed as an automatic behavior triggered by discourse structure, rather than reflecting online changes in word accessibility for either interlocutor, be it for the speaker (Listener-neutral explanation) or for the listener (Listener-modeling explanation). The Repetition Effect in speech production in this dissertation will be accounted for within an exemplar model of the perception/production loop. However, in representational gestures, a much less conventionalized form of performance compared to speech, the Repetition Effect shows a different pattern. When speakers only operate by their assumption of the state of knowledge of the listener, without dynamic, appreciable listener feedback, they steadily reduce most types of representational gesture across tellings. Based on these results, it can be argued that representational gestures primarily serve as a part of speech production, rather than as communicative acts. That is, they are produced without regard to the novelty of the information to the listener, thus, consistent with the Listener-neutral explanation.

Experiência de escrita potencializada pelo gesto. Gesto desdobrado em vibrações que ferem o espírito de forma incontornável. Contorno do acumulam nas mãos, resumidos aos olhos. Uma ilha onde se força o exílio, apreende o gesto e protela o intraduzível. Diferente da língua gestual, do código de acontecimento com margens e paredes, a ferida no sótão, na praia, na montanha. Ocupar as feridas que me ferem com suas respectivas roupagens, marcar o gesto com roupa, alimento, utensílio. Refratariamente, para ambos os lados: os gestos se multiplicam ao infinito, digeridos pelas entranhas e se gesto - a agressividade da marca, do que escorre como sangue, da mão morta, da epilepsia. / Writing experience increased by the gesture. Gesture deployed into vibrations that hurt the spirit so compelling. Outline of the event with the banks and walls, the wound in the attic, on the beach, the mountains. Occupy the wounds that hurt me with outfits, to mark the gesture with clothing, food and devices. Refractory to both sides: the gestures are multiplied to infinity, digested in the bowels and accumulate in the hands, as summarized in the eyes. Islands of a forced exile, seize the gesture and delays the untranslatable. Unlike sign language, the code of gesture – the aggressiveness of the brands, that which drips like blood, the dead hand, the epilepsy.

abstract: The International Standards Organization (ISO) documentation utilizes Fitts’ law to determine the usability of traditional input devices like mouse and touchscreens for one- or two-dimensional operations. To test the hypothesis that Fitts’ Law can be applied to hand/air gesture based computing inputs, Fitts’ multi-directional target acquisition task is applied to three gesture based input devices that utilize different technologies and two baseline devices, mouse and touchscreen. Three target distances and three target sizes were tested six times in a randomized order with a randomized order of the five input technologies. A total of 81 participants’ data were collected for the within subjects design study. Participants were instructed to perform the task as quickly and accurately as possible according to traditional Fitts’ testing procedures. Movement time, error rate, and throughput for each input technology were calculated. Additionally, no standards exist for equating user experience with Fitts’ measures such as movement time, throughput, and error count. To test the hypothesis that a user’s experience can be predicted using Fitts’ measures of movement time, throughput and error count, an ease of use rating using a 5-point scale for each input type was collected from each participant. The calculated Mean Opinion Scores (MOS) were regressed on Fitts’ measures of movement time, throughput, and error count to understand the extent to which they can predict a user’s subjective rating. / Dissertation/Thesis / Masters Thesis Applied Psychology 2015

A presente tese traduz, imagina e transcria uma escola biografemática do gesto denominada Libriação. Este nome significa e deriva de Libras e de criação e vive por meio de biografemas (vidas de grafemar) curricularesdidáticas da arte, do gesto, do romance, do cenalário e da diferença. O texto compõe-se de procedimentos tradutórios de pensar-criar, desde signos e conceitos sentidos; funciona em elementos-atributos científicos-filosóficosartísticos; e traduz em três formas: de pesquisa, ensino e extensão, que não se separam, mas operam, de forma conjunta e transversalizada, em meio à educação e à vida. O primeiro bloco é formado pela biografemática de pesquisar-ensinar-extensionar sobre produções-efeitos de pensamentos e extensões educacionais-contemporâneas; o segundo bloco é escrito por cenalários da educação contemporânea; e o terceiro conclui em um eu pesquisador, um educador e um extensionador, impossíveis de ser definido, pois é resultante de possibilidades diárias-vivas de criar. / The present thesis translates, imagines and transcrires a biographical school of the gesture denominated Libriação. This name means and derives from Libras and creation and lives by means of biografemas (live of grafemar) curricular-didactics of the art, the gesture, the novel, the scenario and the difference. The text consists of translational procedures of thinkingcreating, from signs and concepts sense; Works on scientific-philosophicalartistic- attribute elements; and translates into three forms: research, teaching and extension, which do not separate, but operate, jointly and transversally, in the midst of education and life. The first block is formed by the biographemática of research-teach-extend about productions-effects of thoughts and educational extensions-contemporary; the second block is written by contemporary education cenarios; and the third concludes in a researcher, an educator and an extension worker, impossible to be defined, because it is the result of daily living possibilities to create.