Gesture-based interaction has long been seen as a natural means of input for electronic presentation systems; however, gesture-based presentation systems have not been evaluated in real-world contexts, and the implications of this interaction modality are not known. This thesis describes the design and evaluation of Maestro, a gesture-based presentation system which was developed to explore these issues. This work is presented in two parts. The first part describes Maestro's design, which was informed by a small observational study of people giving talks; and Maestro's evaluation, which involved a two week field study where Maestro was used for lecturing to a class of approximately 100 students. The observational study revealed that presenters regularly gesture towards the content of their slides. As such, Maestro supports several gestures which operate directly on slide content (e.g., pointing to a bullet causes it to be highlighted). The field study confirmed that audience members value these content-centric gestures. Conversely, the use of gestures for navigating slides is perceived to be less efficient than the use of a remote. Additionally, gestural input was found to result in a number of unexpected side effects which may hamper the presenter's ability to fully engage the audience. The second part of the thesis presents a gesture recognizer based on discrete hidden Markov models (DHMMs). Here, the contributions lie in presenting a feature set and a factorization of the standard DHMM observation distribution, which allows modeling of a wide range of gestures (e.g., both one-handed and bimanual gestures), but which uses few modeling parameters. To establish the overall robustness and accuracy of the recognition system, five new users and one expert were asked to perform ten instances of each gesture. The system accurately recognized 85% of gestures for new users, increasing to 96% for the expert user. In both cases, false positives accounted for fewer than 4% of all detections. These error rates compare favourably to those of similar systems.

Gesture-based interaction has long been seen as a natural means of input for electronic presentation systems; however, gesture-based presentation systems have not been evaluated in real-world contexts, and the implications of this interaction modality are not known. This thesis describes the design and evaluation of Maestro, a gesture-based presentation system which was developed to explore these issues. This work is presented in two parts. The first part describes Maestro's design, which was informed by a small observational study of people giving talks; and Maestro's evaluation, which involved a two week field study where Maestro was used for lecturing to a class of approximately 100 students. The observational study revealed that presenters regularly gesture towards the content of their slides. As such, Maestro supports several gestures which operate directly on slide content (e.g., pointing to a bullet causes it to be highlighted). The field study confirmed that audience members value these content-centric gestures. Conversely, the use of gestures for navigating slides is perceived to be less efficient than the use of a remote. Additionally, gestural input was found to result in a number of unexpected side effects which may hamper the presenter's ability to fully engage the audience. The second part of the thesis presents a gesture recognizer based on discrete hidden Markov models (DHMMs). Here, the contributions lie in presenting a feature set and a factorization of the standard DHMM observation distribution, which allows modeling of a wide range of gestures (e.g., both one-handed and bimanual gestures), but which uses few modeling parameters. To establish the overall robustness and accuracy of the recognition system, five new users and one expert were asked to perform ten instances of each gesture. The system accurately recognized 85% of gestures for new users, increasing to 96% for the expert user. In both cases, false positives accounted for fewer than 4% of all detections. These error rates compare favourably to those of similar systems.

O sistema de TV digital interativa está em fase de implantação no Brasil. O middleware Ginga, responsável por permitir a apresentação de programas interativos, prevê que usuários possam interagir com aplicações apresentadas na TV ao pressionar de teclas em um controle remoto. Considerando que controles remotos tradicionais apresentam limitações de usabilidade, este trabalho teve o objetivo investigar a aplicação de conceitos de computação ubíqua, em particular interfaces naturais e multimodais, como alternativas para prover interatividade entre usuários e programas de TV digital. Como resultado, um dispositivo móvel alternativo ao controle remoto tradicional foi utilizado no projeto de novos mecanismos de interação que incluem interfaces baseadas em telas sensíveis ao toque, interfaces sensíveis a gestos capturados por dispositivos que contêm acelerômetros, e interfaces que contêm microfones que permitem entrada de dados por voz. A construção de protótipos correspondentes foi beneficiada pela (assim como beneficiou) implementação prévia de um componente que oferece funcionalidades para envio de dados multimodais para um receptor de TV digital contendo o middleware Ginga, e de um componente que, instalado no receptor, permite a comunicação peer-to-peer entre dispositivos sem fio / The tradicional Brazilian TV system is being replaced by an interactive digital platform. The Ginga middleware, responsible for allowing the presentation of interactive programs, is able to support user interactions with TV applications by means of key presses on a remote control. Since traditional remotes have usability limitations, this work aimed at investigating the application of ubiquitous computing concepts, such as natural and multimodal interfaces, to provide alternatives for the interaction among users and TV applications. Considering the availability of mobile devices such as smartphones, prototype interfaces based on touch screens, as well as gesture-based, accelerometer-based, and voice-based interfaces have been designed and implemented to allow the interaction usually provided by remote controls. The implementation of those interfaces was supported by the previous development of components providing multimodal interaction and peer-to-peer communication in the context of the Brazilian interactive digital TV system middleware

O sistema de TV digital interativa está em fase de implantação no Brasil. O middleware Ginga, responsável por permitir a apresentação de programas interativos, prevê que usuários possam interagir com aplicações apresentadas na TV ao pressionar de teclas em um controle remoto. Considerando que controles remotos tradicionais apresentam limitações de usabilidade, este trabalho teve o objetivo investigar a aplicação de conceitos de computação ubíqua, em particular interfaces naturais e multimodais, como alternativas para prover interatividade entre usuários e programas de TV digital. Como resultado, um dispositivo móvel alternativo ao controle remoto tradicional foi utilizado no projeto de novos mecanismos de interação que incluem interfaces baseadas em telas sensíveis ao toque, interfaces sensíveis a gestos capturados por dispositivos que contêm acelerômetros, e interfaces que contêm microfones que permitem entrada de dados por voz. A construção de protótipos correspondentes foi beneficiada pela (assim como beneficiou) implementação prévia de um componente que oferece funcionalidades para envio de dados multimodais para um receptor de TV digital contendo o middleware Ginga, e de um componente que, instalado no receptor, permite a comunicação peer-to-peer entre dispositivos sem fio / The tradicional Brazilian TV system is being replaced by an interactive digital platform. The Ginga middleware, responsible for allowing the presentation of interactive programs, is able to support user interactions with TV applications by means of key presses on a remote control. Since traditional remotes have usability limitations, this work aimed at investigating the application of ubiquitous computing concepts, such as natural and multimodal interfaces, to provide alternatives for the interaction among users and TV applications. Considering the availability of mobile devices such as smartphones, prototype interfaces based on touch screens, as well as gesture-based, accelerometer-based, and voice-based interfaces have been designed and implemented to allow the interaction usually provided by remote controls. The implementation of those interfaces was supported by the previous development of components providing multimodal interaction and peer-to-peer communication in the context of the Brazilian interactive digital TV system middleware