In this thesis we report on remote interaction techniques for horizontal and vertical large displays. For vertical large displays, we present MultiPoint, a set of perspective-based remote pointing techniques that allows users to perform bimanual and multi-finger remote manipulation of graphical objects on large displays. We conducted two empirical studies that compared remote pointing techniques performed using fingers and laser pointers, in single and multi-finger pointing interactions. The MultiPoint techniques were found suitable for interacting with vertical large displays. We explore techniques that are well suited for exclusively single-point use cases as well as for multipoint scenarios. With Pointable, we also explored perspective-based pointing with in-air gestures, but in a tabletop scenario. We conducted 3 experiments; the first showed that pointing at a distance using Pointable has a Fitts’ law throughput comparable to that of a mouse. In the second experiment, we found that Pointable had the same performance as multi-touch input in a resize, rotate and drag task. In a third study, we observed that when given the choice, over 75% of participants preferred to use Pointable over multi-touch for target manipulation. In general, Pointable allowed users to manipulate out-of-reach targets, without loss of performance, while minimizing the need to lean, stand up, or involve collocated collaborators. / Thesis (Master, Computing) -- Queen's University, 2012-06-08 11:59:53.418

Manipulating 3D objects on a tabletop computer is inherently problematic. The flat surface of tabletop computers enable natural 2D interaction, but lack the additional dimension needed to intuitively facilitate 3D object manipulation. In this thesis I present TZee, a passive tangible widget that enables natural interactions with 3D objects by exploiting the lighting properties of diffuse illumination (DI) multi-touch tabletops. The Tangible User Interface (TUI), TZee is constructed from several pieces of stacked acrylic glass. The stacked glass enables TZee to channel the light emitted from the tabletop slightly higher above the surface without major light loss. This technique allows the tangible interface to transmit touches on the device to the tabletop without any supplementary power. TZee enables simple translation, rotation and scaling along the x, y, or z axes. This thesis discusses several important design considerations of TZee, demonstrated TZee’s value though several applications and a gesture design study.

Manipulating 3D objects on a tabletop computer is inherently problematic. The flat surface of tabletop computers enable natural 2D interaction, but lack the additional dimension needed to intuitively facilitate 3D object manipulation. In this thesis I present TZee, a passive tangible widget that enables natural interactions with 3D objects by exploiting the lighting properties of diffuse illumination (DI) multi-touch tabletops. The Tangible User Interface (TUI), TZee is constructed from several pieces of stacked acrylic glass. The stacked glass enables TZee to channel the light emitted from the tabletop slightly higher above the surface without major light loss. This technique allows the tangible interface to transmit touches on the device to the tabletop without any supplementary power. TZee enables simple translation, rotation and scaling along the x, y, or z axes. This thesis discusses several important design considerations of TZee, demonstrated TZee’s value though several applications and a gesture design study.

Multi-touch tablets allow users to interact with computers through intuitive, natural gestures and direct manipulation of digital objects. One advantage of these devices is that they can offer a large, collaborative space where several users can work on a task at the same time. However the lack of privacy in these situations makes standard password-based authentication easily compromised. This work presents a new gesture-based authentication system based on users' unique signature of touch motion. This technique has two key features. First, at each step in authentication the system prompts the user to make a specific gesture selected to maximize the expected long-term information gain. Second, each gesture is integrated using a hierarchical probabilistic model, allowing the system to accept or reject a user after a variable number of gestures. This touch-based approach would allow the user to accurately authenticate without the need to cover their hand or look over their shoulder. This method has been tested using a set of samples collected under real-world conditions in a business office, with a touch tablet that was used on a near daily basis by users familiar with the device. Despite the lack of sophisticated, high-precision equipment, the system is able to achieve high user recognition accuracy with relatively few gestures, demonstrating that human touch patterns have a distinctive signature" that can be used as a powerful biometric measure for user recognition and personalization.

Musik-applikationen c3n play används på surfplattor och mobiltelefoner som har den kapacitiva pekskärmstypen. I dagsläget kan musik-applikationen enbart styras med fingrarna. Spelpjäser, utvecklades för att målgruppen skulle kunna kombinera fingerinmatning med något fysiskt och för att underlätta spelandet. En konvergerande produktutvecklingsprocess innefattande faser och aktiviteter tillämpades. Inledningsvis genomfördes en detaljerad informationsinsamling som grund för det fortsatta arbetet. Analys av konkurrenternas produkter och en benchmarking implementerades, för att både identifiera kraven som ställts på produkten och upprätta en kravspecifikation. Spelpjäsen delades in i tre delar: formen på hela spelpjäsen, mönster som gör att användarna kan identifiera varje spelpjäs och mönster på undersidan som programvaran kunde identifiera. Därefter genererades flera idéer på formen av spelpjäsen, vilka utvärderades med hänsyn till de uppsatta kraven. Idéer som ansågs uppfylla kraven bäst gick vidare till konceptgenereringsfasen, där fem olika förslag kunde visualiseras i skisser och CAD-modeller (Computer Aidid Design). Koncepten utvärderades och förbättrades för att tillfredsställa användarnas behov och förväntningar. Det slutliga konceptet på formen gjordes i fyra olika varianter, vilka framställdes som prototyper för vidare test av målgruppen. Konceptlösningar på spelpjäsens identifieringsmönster av användaren analyserades för att passa in i c3n play-användargränssnitten. Tekniken för att identifiera spelpjäsen mot den kapacitiva pekskärmstypen utvecklades via tester, där avstånd och vinklar på kontaktpunkterna bestämdes. Viktiga produktutvecklingsverktyg implementerades för att säkerställa såväl spelpjäsernas hållbarhet som miljövänlighet, med hänsyn till produktlivscykel, tillverkning, montering och tillförlitlighet samt kostnadsmedvetenhet. Därefter valdes lämpliga material samt en lämplig tillverkningsmetod. Detta resulterade i en produktfamilj bestående av åtta stycken spelpjäser som ändrar på var sin funktion i c3n play-applikationen. En spelpjäs som förändrar amplitud, resonans, högtpassfilter, lågt-passfilter, delay, zoom + pan, kompression och feedback. / The application c3n play can be used on tablets and mobile phones that have capacitive multi touchscreens. The application can only be controlled by using the finger input. Tangible user interface was developed so the target group can use something physically “checkers” in a combination with the finger input. A product development process comprising phases and activities was implemented. Initially a detailed information collection were conducted and used as underlying to the continued work. Competitors’ products were analyzed and a benchmarking was implemented to identify user needs and to be able to make a target specification. Checkers was divided in three parts, shape of the whole checker, pattern so users can identify every checker and a pattern on the bottom so the software can identify the checkers on the screen. After that ideas were generated and the shape of the checker was evaluated. Ideas that was considered to meet the user needs went on to concept generation phase, where five different proposals was visualized in sketches and CAD (Computer Aided Design) - models. Concepts was evaluated and improved to satisfy customer needs and expectations. Final concept of the shape was developed into four different variants that was prepared as prototypes for further testing by target group. Concept solutions on identification pattern was analyzed to fit c3n play user interface. Technology to identify the checker against capacitive screen was developed by tests. Where distance and angles of contact points was determined. Important product development tools was implemented to ensure the checkers sustainability. With respect to the product life cycle, environment, manufacturing, assembly, reliability and cost consciousness. Suitable materials and manufacturing method was then chosen. Resulted in a product family consisted of eight different checkers. Each checker can change a different function in the c3n play-application. Amplitude, Resonance, Highpassfilter, Low-passfilter, Delay, Zoom + Pan, Compression and Feedback.

Above-surface interaction is a new and exciting topic in the field of human-computer interaction (HCI). It focuses on the design and evaluation of systems that humans can operate by moving their hands in the space above or in front of interactive displays. While many technologies emerge that make such systems possible, much research is still needed to make this interaction as natural and effortless as possible. First this thesis presents a set of guidelines for designing above-surface interactions, a collection of widgets that were designed based on these guidelines, and a system that can approximate the height of hands above a diffused surface illumination (DSI) device without any additional sensors. Then the thesis focuses on interaction techniques for activating graphical widgets located in this above-surface space. Finally, it presents a pair of studies that were conducted to investigate item selection in the space above a multitouch surface. The first study was conducted to elicit a set of gestures for above-table widget activation from a group of users. Several gestures were proposed by the designers to be compared with the user-generated gestures. The follow-up study was conducted to evaluate and compare these gestures based on their performance. The findings of these studies showed that there was no clear agreement on what gestures should be used to select objects in mid-air, and that performance was better when using gestures that were chosen less frequently, but predicted to be better by the designers, as opposed to those most frequently suggested by participants.

Research in multi-touch interaction has typically been focused on direct spatial manipulation; techniques have been created to result in the most intuitive mapping between the movement of the hand and the resultant change in the virtual object. As we attempt to design for more complex operations, the effectiveness of spatial manipulation as a metaphor becomes weak. We introduce two new platforms for multi-touch computing: a gesture recognition system, and a new interaction technique. I present Multi-Tap Sliders, a new interaction technique for operation in what we call non-spatial parametric spaces. Such spaces do not have an obvious literal spatial representation, (Eg.: exposure, brightness, contrast and saturation for image editing). The multi-tap sliders encourage the user to keep her visual focus on the tar- get, instead of requiring her to look back at the interface. My research emphasizes ergonomics, clear visual design, and fluid transition between modes of operation. Through a series of iterations, I develop a new technique for quickly selecting and adjusting multiple numerical parameters. Evaluations of multi-tap sliders show improvements over traditional sliders. To facilitate further research on multi-touch gestural interaction, I developed mGestr: a training and recognition system using hidden Markov models for designing a multi-touch gesture set. Our evaluation shows successful recognition rates of up to 95%. The recognition framework is packaged into a service for easy integration with existing applications.

Knowledge workers are frequently called upon to perform deep, critical reading involving a heightened level of interaction with the reading media and other tools. This process, known as active reading, entails highlighting, commenting upon, and flipping through a text, in addition to other actions. While paper is traditionally seen as the ideal medium for active reading, computers have recently become comparable to paper through replicating the latter’s affordances. But even paper is not a panacea; it offers an inflexible document representation that supports some things well, such as embellishment, but supports others very poorly, like comparison and large scale annotation. In response to this, I developed a prototype system, called LiquidText, to embody a flexible, high degree-of-freedom visual representation that seeks to alleviate some of the problems in paper and paper-like representations. To provide efficient control of this representation, LiquidText runs on a multi-finger touch and gesture based platform. To guide the development of this system, I conducted a formative study of current active reading practice. I investigated knowledge workers’ active reading habits, perceptions, and the problems they face with current reading media. I also inquired into what they would like in a future active reading environment. I used these results in conjunction with multiple design iterations and formative system evaluations to refine LiquidText for use in a summative study. The summative study assessed, through a controlled, laboratory evaluation, LiquidText’s impact on 1) the subjective experience of active reading, 2) the process of active reading, and 3) the outputs resulting from active reading. Generally, the study found a strong participant preference for LiquidText, and a focus on the creation of a summary of the original document as part of the reading process. On average, reading outputs were not significantly better or worse with LiquidText, but some conditions were observed that may help identify the subset of people for whom LiquidText will result in an improvement.

Above-surface interaction is a new and exciting topic in the field of human-computer interaction (HCI). It focuses on the design and evaluation of systems that humans can operate by moving their hands in the space above or in front of interactive displays. While many technologies emerge that make such systems possible, much research is still needed to make this interaction as natural and effortless as possible. First this thesis presents a set of guidelines for designing above-surface interactions, a collection of widgets that were designed based on these guidelines, and a system that can approximate the height of hands above a diffused surface illumination (DSI) device without any additional sensors. Then the thesis focuses on interaction techniques for activating graphical widgets located in this above-surface space. Finally, it presents a pair of studies that were conducted to investigate item selection in the space above a multitouch surface. The first study was conducted to elicit a set of gestures for above-table widget activation from a group of users. Several gestures were proposed by the designers to be compared with the user-generated gestures. The follow-up study was conducted to evaluate and compare these gestures based on their performance. The findings of these studies showed that there was no clear agreement on what gestures should be used to select objects in mid-air, and that performance was better when using gestures that were chosen less frequently, but predicted to be better by the designers, as opposed to those most frequently suggested by participants.

MIMICS is an acronym for Multitouch Interface as a MIDI Control Suite. Multitouch interfaces are well known for their softer learning curve compared with classical computer graphic interfaces. Therefore, its application in music interfaces may solve some of the actual limitations of hardwarebased and software-based devices. The multitouch solution is based on an optical effect called Frustrated Total Internal Reflection which is forced by using infrared light sources. This effect allows finger tracking using computer vision engines. In the thesis we describe a low-cost and affordable Hardware/Software solution for MIDI-based devices control from a functional multitouch environment. The MIMICS is presented to the user as a rear projected 30 inches screen containing a graphical interface adapted for multitouch. Implementation of several ‘ready-to-play’ applications is demonstrated that emulate classical and new MIDI control devices. The report also contains an evaluation of the results demonstrating that MIMICS is suitable for life-oriented music performances as well as for recording studio works.