Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 2012

The number of devices with touch input such as smart phones, computers, and tablets has grown extensively in recent years. Native applications on these devices have access to this touch and gesture information and can provide a rich, interactive experience. Web applications, however, lack a consistent and uniform way to retrieve touch and gesture input. With the quality and robustness of web applications continually growing and replacing native applications in many areas, a way to access and harness touch input is critical. This paper proposes two JavaScript libraries that provide a reliable and easy way for web applications to use touch efficiently and effectively. First, getTjs abstracts the gathering of touch events for most mobile and desktop touch devices. GenGesjs, the second library, receives this information and identifies gestures based on the touch input. Web applications can have this gesture information pushed to them as it is received or instead request the most recent gestures when desired. An example of interfacing with both libraries is provided in the form of WebMol. WebMol is a web application that allows for three dimensional viewing of molecules using WebGL. Gestures from GenGesjs are translated to interactions with the molecules, providing an intuitive interface for users. Using both of these libraries, web applications can easily tap into touch input resulting in an improved user experience regardless of the device. / text

Storytelling is a powerful means of communication that has been employed by humankind from the early stages of development. As technology has advanced, the medium through which people tell stories has evolved from verbal, to writing, performing on stage, and more recently television, movies, and video games. A promising medium for the telling of stories in an in-person, one-on-one or one-to-many setting is a digital table—a large, horizontal multi-touch surface—that can provide quick access to visuals and narrative elements at the touch of one’s hands and fingers. In this work, I present the results of an exploratory study on storytellers’ interaction behaviours while working with digital tables, and its physical counterparts of sand and water. My results highlight some of the differences in these media that can both help and hinder a storyteller’s narrative process. I use these findings to present design implications for the design of applications for storytelling on digital multi-touch surfaces.

Storytelling is a powerful means of communication that has been employed by humankind from the early stages of development. As technology has advanced, the medium through which people tell stories has evolved from verbal, to writing, performing on stage, and more recently television, movies, and video games. A promising medium for the telling of stories in an in-person, one-on-one or one-to-many setting is a digital table—a large, horizontal multi-touch surface—that can provide quick access to visuals and narrative elements at the touch of one’s hands and fingers. In this work, I present the results of an exploratory study on storytellers’ interaction behaviours while working with digital tables, and its physical counterparts of sand and water. My results highlight some of the differences in these media that can both help and hinder a storyteller’s narrative process. I use these findings to present design implications for the design of applications for storytelling on digital multi-touch surfaces.

Multi-touch technology, used in consumer products such as the iPad, enables users to register multiple points of contact at the same time; this enables a user to interact with a touch screen interface using several fingers on one hand, or even both hands. This affords interface designers the opportunity to define gestural interactions based on what is most natural for users and not on merely what can be recognised and processed by technology. In light of this, the research question that this study aimed to address was: what is the most intuitive user-defined gesture set for conducting an informational search on a multi-touch tablet web browser? In addressing this research question, the aim of this study was to create a user-defined gesture set for conducting an informational search on a multi-touch tablet web browser, based on gestures elicited from participants with little or no experience with touch screen devices. It was necessary to use these participants as users who are familiar with touch screen interfaces would draw upon the gestures they have learnt or used before, and would therefore be biased in the gestures they proposed. Inexperienced or naïve users would simply provide gestures that came naturally to them, providing a more accurate reflection of what a typical, unbiased user would do. A set of hypotheses, relating to the gestures that would be elicited from this participant group, were drawn up and investigated. These investigations yielded the following key findings: • The use of two-handed gestures should be limited. • If two-handed gestures are developed for a specific function, an alternative one-handed gesture should be made available. • It is not be advisable to create completely novel gestures for tablet web browsing that do not correspond to any of the ways in which desktop web browsing is performed. • Should novel gestures be developed for tablet web browsing, gestures that are desktop computing adaptations, including those that require menu access, should be made available as alternatives to users. • Tasks should be designed is such a way that they may be completed with a variety of gestures. • Complex tasks should be designed in such a way that they may be achieved through varying combinations of gestures. These findings may assist interface designers and developers in the gestures they design or developer for their applications. In addition to these findings, the study presents a coherent, user-defined gesture set that may be used in practice by designers or developers.

We are studying the use of tilt-enabled handheld touchscreen devices as an interface for top-down strategy games. We will explore how using different input modes (tilt and touch) compare for certain tasks in terms of efficiency and comfort. Real-time and turn-based strategy games are a popular form of electronic gaming, though these games currently have only minor representation on tablets. This genre of game requires both a wide variety of input and the display of a wealth of information. We are exploring whether, with suitable interface developments, this genre can become as accessible on tablet devices as on traditional computers. These interface approaches may also prove useful for expanding the presence of other game genres in the mobile space.

Interactive tabletop systems are becoming popular platforms for group activities. However, current common tabletops do not provide capabilities to differentiate interactions among simultaneous users, i.e. to associate a touch point with its proper owner. My thesis proposes and explores the use of an important biometric property of users as the basis for touch discrimination on multi-user tabletops: Finger Orientation (FO). In this thesis, I first collect the FO ranges of users standing in different positions around a tabletop. Second, I implement a system that uses FO to determine where the users are standing, and based on that extrapolate the owner of the touch. Next, I evaluate the system with two separate experiments, present the results, and discuss all findings. Furthermore, I explore some enhancements with a simple quantitative study. My results indicate that finger orientation is a good natural biometric trait enhances multi-user recognition on tabletops.

Interactive tabletop systems are becoming popular platforms for group activities. However, current common tabletops do not provide capabilities to differentiate interactions among simultaneous users, i.e. to associate a touch point with its proper owner. My thesis proposes and explores the use of an important biometric property of users as the basis for touch discrimination on multi-user tabletops: Finger Orientation (FO). In this thesis, I first collect the FO ranges of users standing in different positions around a tabletop. Second, I implement a system that uses FO to determine where the users are standing, and based on that extrapolate the owner of the touch. Next, I evaluate the system with two separate experiments, present the results, and discuss all findings. Furthermore, I explore some enhancements with a simple quantitative study. My results indicate that finger orientation is a good natural biometric trait enhances multi-user recognition on tabletops.

Multi-touch technology and digital tabletops have been used in different fields. They provide a natural way of interaction with computers through gestures. In this report, we present a digital desk for power plant control rooms aimed at visualizing the power grid status. The thesis reports about the development of this desk that comprised field studies, use cases and requirements identification, low fidelity prototyping, and software development. A final evaluation of the design indicated that digital tabletops can be valuable for control room operators since they can enhance learning and communication among the collaborating operators. The work was done at ABB Corporate Research in Sweden.

Today multi-touch technology is the basis for many new techniques designed to improveinteractions with computers and mobile devices. It seems that multi-touch screen interfacemakes the user handling very natural in the sense that there is no need for a manual in how tointeract with the object on the screen.The aim with this paper is to establish a fundamental set of design principles intendedspecifically for large multi-touch interfaces. To reach this goal we have implemented a coupleof sub-goals beforehand:It was essential that we acquired a good understanding of the current state of the multi-touchinterface and the different implementations that exist today. To make this possible weconstructed a multi-touch display, "Rosie". Knowing how the hardware is produced today willhelp us understand the limitations and also the possibilities of the design implementationstoday and in the future.We also needed to devise a sound interaction design process that conveys the moderndesigners work. During this design process four methods were implemented that gave usdeeper understanding how to reach the result in this paper (design principles). The methodsare: Qualitative conceptualisation, Qualitative user-testing, Participatory design, and Iterativeprototyping. Doing these methods we gained knowledge through the process and experienceof for example, building, running workshops, doing video-prototypes and etc. Creative designwas very relevant in our design process.The result in this paper is a foundation for a set of design principles with relevance for multitouchinterfaces and a interesting design process for developing multi-touch applications.