This thesis investigates the impact of different factors on public displays that use touch and mid-air gestures. We present a novel application for public displays: MirrorTouch — a game combining touch interaction with mid-air gesturing through depth sense. We studied the impact of these factors through a series field studies. We show that clear affordances are important to communicate both modalities clearly and also how to improve the conversion rate of passers-by. We present our findings of how social situations around such a display varied with the location where the display was placed. We also present quantitative data on group behavior around our public display, such as a measuring of the honeypot effect and how people made transitions between the different modalities. Our results can be of great value to designers and researchers of public displays that want to deploy displays with similar techniques.

This interaction design thesis aimed to explore how users could be introduced to the different functionalities of a gesture-based touch screen interface. This was done through a user-centred design research process where the designer was taught different artefacts by experienced users. Insights from this process lay the foundation for an interactive, digital gesture-introduction prototype.Testing said prototype with users yielded this study's results. While containing several areas for improvement regarding implementation and behaviour, the prototype's base methods and qualities were well received. Further development would be needed to fully assess its viability. The user-centred research methods used in this project proved valuable for later ideation and prototyping stages. Activities and results from this project indicate a potential for designers to further explore the possibilities for ensuring the discoverability of touch-gesture interactions. For future projects the author suggests more extensive research and testing using a greater sample size and wider demographic.

No description available.

La version intégrale de cette thèse est disponible uniquement pour consultation individuelle à la Bibliothèque de musique de l’Université de Montréal (www.bib.umontreal.ca/MU). / Cette thèse présente la démarche compositionnelle relative à six œuvres de musique mixte interactive pour hyper-flûtes, soit des flûtes traversières munies de capteurs gestuels électroniques reliés à un ordinateur. Le jeu instrumental est ainsi augmenté par des composantes électroacoustiques qui peuvent être contrôlées par l’interprète. Une grande partie de cet ouvrage est consacrée à la recherche autour de ces œuvres. On peut y lire un bref historique de la musique mixte qui souligne l'apport de la flûte à la musique électroacoustique. On se penche sur les défis de synchronisation du son instrumental et électronique auxquels font face les compositeurs depuis les années 1950, pour ensuite déterminer en quoi consiste la composition interactive. Après avoir défini la notion de geste instrumental, en particulier en ce qui concerne le jeu de la flûte traversière, on aborde les détails de la lutherie électronique de l’hyper-flûte et de l’hyper-flûte-basse. Ces descriptions techniques sont précédées de commentaires sur différents instruments augmentés et spécifiquement sur les différentes flûtes connues. Cette partie se termine par un exposé concernant l’interaction et les stratégies de mappage des données gestuelles vers les correspondances sonores. La suite est consacrée aux compositions. On peut d’abord y lire mon point de vue sur la composition interactive et l’improvisation. Six chapitres décrivent ensuite en détail chacune des œuvres. On y présente les sources d’inspiration et une analyse formelle sommaire de chaque pièce, puis on élabore sur les composantes instrumentales et électroacoustiques, en explicitant les types d’interactions utilisés. Soulignons que quatre de ces œuvres ont été élaborées autour de textes poétiques. Finalement, une courte conclusion résume les objectifs artistiques réalisés dans le cadre de ce doctorat et les pistes envisagées pour la poursuite de mon travail créateur avec les hyper-flûtes. / This dissertation presents the compositional processes of six pieces with live interactive electronics for hyper-flutes: flutes equipped with electronic motion sensors connected to a computer. Instrumental techniques are thus augmented by electroacoustic components that can be controlled by the performer. A significant portion of this document is focussed on the research framing these works. A brief history of «musique mixte» is presented, underlining the contribution of the flute to electroacoustic music. The challenges associated with the synchronization of instrumental and electronic sounds with which composers have been faced since the 1950’s are then addressed, providing the context for establishing the essential nature of interactive composition. After having defined the notion of instrumental gesture, particularly as it concerns flute techniques, details of the electronic design of the hyper-flute and the hyper-bass-flute are considered. The descriptions of these techniques are prefaced by observations about other augmented instruments and more specifically the different known flutes. This part concludes with a presentation of the interaction and mapping strategies of gestural data as it relates to the corresponding audio elements. The sections that follow focus attention on the compositions. I begin by elaborating my own perspective on interactive composition and improvisation. Six chapters then provide detailed descriptions of each of the pieces, starting with the sources of inspiration and a brief formal analysis. I go on to explain their instrumental and electroacoustic components, detailing the types of interactions used. Of particular note is the fact that four of these works were developed based on poetic texts. Lastly, a short conclusion summarizes the artistic objectives realized during my doctoral tenure as well as the potential opportunities for the continuation of my creative work with the hyper-flutes.

The dissemination of new innovative technology requires feasibility and simplicity. The problem with marker-based augmented reality is similar to glove-based hand gesture recognition: they both require an additional component to function. This thesis investigates the possibility of combining markerless augmented reality together with appearance-based hand gesture recognition by implementing a game with good user experience. The methods employed in this research consist of a game implementation and a pre-study meant for measuring interactive accuracy and precision, and for deciding upon which gestures should be utilized in the game. A test environment was realized in Unity using ARKit and Manomotion SDK. Similarly, the implementation of the game used the same development tools. However, Blender was used for creating the 3D models. The results from 15 testers showed that the pinching gesture was the most favorable one. The game was evaluated with a System Usability Scale (SUS) and received a score of 70.77 among 12 game testers, which indicates that the augmented reality game, which interaction method is solely based on bare-hands, can be quite enjoyable.