In this thesis we report on remote interaction techniques for horizontal and vertical large displays. For vertical large displays, we present MultiPoint, a set of perspective-based remote pointing techniques that allows users to perform bimanual and multi-finger remote manipulation of graphical objects on large displays. We conducted two empirical studies that compared remote pointing techniques performed using fingers and laser pointers, in single and multi-finger pointing interactions. The MultiPoint techniques were found suitable for interacting with vertical large displays. We explore techniques that are well suited for exclusively single-point use cases as well as for multipoint scenarios. With Pointable, we also explored perspective-based pointing with in-air gestures, but in a tabletop scenario. We conducted 3 experiments; the first showed that pointing at a distance using Pointable has a Fitts’ law throughput comparable to that of a mouse. In the second experiment, we found that Pointable had the same performance as multi-touch input in a resize, rotate and drag task. In a third study, we observed that when given the choice, over 75% of participants preferred to use Pointable over multi-touch for target manipulation. In general, Pointable allowed users to manipulate out-of-reach targets, without loss of performance, while minimizing the need to lean, stand up, or involve collocated collaborators. / Thesis (Master, Computing) -- Queen's University, 2012-06-08 11:59:53.418

Advances in display technology are increasing screen size and reducing their costs. Large, wall sized screens can be found in malls, public squares and office buildings displaying photos, videos, advertising, news and announcements. The audience of large displays in public, urban environments can be constantly changing and in this thesis we investigate applying automated layout algorithms to large displays to adapt layouts to changing audiences. Abstract We review current automated layouts, discuss new challenges when applying automated layouts to large screens and present our design space for automated layouts. We investigate the human factors (such as attention, vision, and perception) and the system factors (such as display space and input devices) involved in automated layout for large displays. Abstract This thesis presents the Viewer Aware Layout (VAL) system. VAL is a system that automates the layout of images on a large display and is designed to allow many users to collectively view images. VAL, and features of VAL, are tested for engagement and it is found that there is a significant increase in agreement by users with statements describing an engaging experience when VAL is applied to an image layout. This thesis concludes with a summary of the research contributions and proposes future areas of work.

Some computer users employ large displays, 6 or more monitors, in order to view a large amount of data on a single desktop at one time.  This layout can be useful when the user is performing tasks in which they must view several different information sources at a time.  For example, a user may be writing a paper in which they may be simultaneously typing a document, reading another paper, and view a spreadsheet.  After the task is completed, the user may close all of the windows, however sometime later they may want to view a document associated with that task.  A possible scenario is for the user to know that they were viewing an important document in their top left monitor, but they cannot remember which document.  SpatialHistory looks to allow a user to recall which windows and documents were open at a certain time spatially.  The user may query a particular region of a large display and SpatialHistory will report the windows that were open in that area.  Through a user study, we conclude that i) some users organize their large displays in a spatial manner placing certain types of documents and windows in certain places and that ii) our tool has the potential to help users recall previously viewed windows based on a spatial memory of their desktop. / Master of Science

Display technology has developed significantly over the last decade, and it is becoming increasingly feasible to construct large, high-resolution displays. Prior work has shown a number of key performance advantages for these displays that can largely be attributed to the replacement of virtual navigation (e.g., panning and zooming) with physical navigation (e.g., moving, turning, glancing). This research moves beyond the question of performance or efficiency and examines ways in which the large, high-resolution display can support the cognitive demanding task of sensemaking. The core contribution of this work is to show that the physical properties of large, high- resolution displays create a fundamentally different environment from conventional displays, one that is inherently spatial, resulting in support for a greater range of embodied resources. To support this, we describe a series of studies that examined the process of sensemaking on one of these displays. These studies illustrate how the display becomes a cognitive partner of the the analyst, encouraging the use of the space for the externalization of the analyst's thought process or findings. We particularly highlight how the flexibility of the space sup- ports the use of incremental formalism, a process of gradually structuring information as understanding grows. Building on these observations, we have developed a new sensemaking environment called Analyst's Workspace (AW), which makes use of a large, high-resolution display as a core component of its design. The primary goal of AW is to provide an environment that unifies the activities of foraging and synthesis into a single investigative thread. AW addresses this goal through the use of an integrated spatial environment in which full text documents serve as primary sources of information, investigative tools for pursuing leads, and sensemaking artifacts that can be arranged in the space to encode information about relationships between events and entities. This work also provides a collection of design principles that fell out of the development of AW, and that we hope can guide future development of analytic tools on large, high-resolution displays. / Ph. D.

Geospatial intelligence analysts, epidemiologists, sociologists, and biologists are all faced with trying to understand massive datasets that require integrating spatial and multidimensional data. Information visualizations are often used to aid these scientists, but designing the visualizations is challenging. One aspect of the visualization design space is a choice of when to use a single complex integrated view and when to use multiple simple views. Because of the many tradeoffs involved with this decision it is not always clear which design to use. Additionally, as the cost of display technologies continues to decrease, large, high resolution displays are gradually becoming a more viable option for single users. These large displays offer new opportunities for scaling up visualization to very large datasets. Visualizations that are visually scalable are able to effectively display large datasets in terms of both graphical scalability (the number of pixels required) and perceptual scalability (the effectiveness of a visualization, measured in terms of user performance, as the amount of data being visualized is scaled-up). The purpose of this research was to compare information visualization designs for integrating spatial and multidimensional data in terms of their visual scalability for large, high resolution displays. Toward that goal a hierarchical design space was articulated and a series of user experiments were performed. A baseline was established by comparing user performance with opposing visualizations on a desktop monitor. Then, visualizations were compared as more information was added using the additional pixels available with a large, high resolution display. Results showed that integrated views were more visually scalable than multiple view visualizations. The visualizations tested were even scalable beyond the limits of visual acuity. User performance on certain tasks improved due to the additional information that was visualized even on a display with enough pixels to require physical navigation to visually distinguish all elements. The reasons for the benefits of integrated views on large, high resolution displays include a reduction in navigation due to spatial grouping and visual aggregation resulting in the emergence of patterns. These findings can help with the design of information visualizations for large, high resolution displays. / Ph. D.

Large displays and information kiosks are becoming increasingly common installations in public venues to provide an efficient self-serve means for patrons to access information and/or services. They have evolved over a relatively short period of time from non-digital, non-interactive static displays to more elaborate media-rich digital interactive systems. While the content and purposes of kiosks have changed, they are still largely based on the traditional single-user-driven design paradigm despite the fact that people often venture to these venues in small social groups, i.e., with family and/or friends. This often limits how groups collaborate and forces transactions to be serialized. This thesis explores design constraints for interaction by multiple social groups in parallel on shared large vertical displays. To better understand design requirements for these systems, this research is separated into two parts: a preliminary observational field study and a follow-up controlled study. Using an observational field study, fundamental patterns of how people use existing public displays are studied: their orientation, positioning, group identification, and behaviour within and between social groups just-before, during, and just-after usage. These results are then used to motivate a controlled experiment where two individuals or two pairs of individuals complete tasks concurrently on a low-fidelity large vertical display. Results from the studies demonstrate that vertical surface territories are similar to those found in horizontal tabletops in function, but their definitions and social conventions are different. In addition, the nature of use-while-standing systems results in more complex and dynamic physical territories around the display. We show that the anthropological notion of personal space must be slightly refined for application to vertical displays.

Large displays and information kiosks are becoming increasingly common installations in public venues to provide an efficient self-serve means for patrons to access information and/or services. They have evolved over a relatively short period of time from non-digital, non-interactive static displays to more elaborate media-rich digital interactive systems. While the content and purposes of kiosks have changed, they are still largely based on the traditional single-user-driven design paradigm despite the fact that people often venture to these venues in small social groups, i.e., with family and/or friends. This often limits how groups collaborate and forces transactions to be serialized. This thesis explores design constraints for interaction by multiple social groups in parallel on shared large vertical displays. To better understand design requirements for these systems, this research is separated into two parts: a preliminary observational field study and a follow-up controlled study. Using an observational field study, fundamental patterns of how people use existing public displays are studied: their orientation, positioning, group identification, and behaviour within and between social groups just-before, during, and just-after usage. These results are then used to motivate a controlled experiment where two individuals or two pairs of individuals complete tasks concurrently on a low-fidelity large vertical display. Results from the studies demonstrate that vertical surface territories are similar to those found in horizontal tabletops in function, but their definitions and social conventions are different. In addition, the nature of use-while-standing systems results in more complex and dynamic physical territories around the display. We show that the anthropological notion of personal space must be slightly refined for application to vertical displays.

Geospatial visualizations are becoming a larger part of society. From using maps to go from one location to another to using battlefield visualizations to help the military, geospatial visualizations are becoming a larger part of people's lives. At the same time, large displays are becoming more prominent in people's lives. From large fifty-monitor tiled displays to dual monitor desktop systems people are using larger displays more often in their daily lives. This dissertation summarizes our work with large displays and geospatial visualizations. We show dramatic increases in performance of more than ten times performance improvement when using larger displays that offer a greater number of pixels. We show performance improvements for a range of tasks from simple navigation to complex pattern finding tasks. This dissertation contributes to the fields of human-computer interaction and information visualization in that it shows performance improvements as analytical force multipliers and explains why such performance exists. It explains how virtual navigation (mouse and keyboard input) correlates to physical navigation (body movement) to explain performance improvements. In addition, this dissertation explains how semantic zooming, space scale, task scale, and task type all are variables that influence human behavior in both navigation and performance. This dissertation addresses primarily geospatial information visualizations, but extends to other generic spatially oriented visualizations. The impacts of large displays for both geospatial information visualizations and generic spatially oriented visualizations are explained. / Ph. D.

As increasingly large displays are integrated into personal workspaces, mouse-based interaction becomes more problematic. Users must repeatedly "clutch" the mouse for long distance movements [61]. The visibility of the cursor is also problematic in large screens, since the percentage of the screen space that the cursor takes from the whole display gets smaller. We test multi-scale approaches to mouse interaction that utilize dynamic speed and size techniques to grow the cursor larger and faster for long movements. Using Fitts' Law methods, we experimentally compare different implementations to optimize the mouse design for large displays and to test how they scale to large displays. We also compare them to techniques that integrate absolute pointing with head tracking. Results indicate that with some implementation level modifications the mouse device can scale well up to even a 100 megapixel display with lower mean movement times as compared to integrating absolute pointing techniques to mouse input while maintaining fast performance of the typical mouse configuration on small screens for short distance movements. Designs that have multiple acceleration levels and 4x maximum acceleration reduced average number of clutching to less than one per task in a 100 megapixel display. Dynamic size cursors statistically improve pointing performance. Results also indicated that dynamic speed transitions should be as smooth as possible without steps of more than 2x increase in speed. / Master of Science

Doctor of Philosophy (PhD) / Large displays are everywhere, and have been shown to provide higher productivity gain and user satisfaction compared to traditional desktop monitors. The computer mouse remains the most common input tool for users to interact with these larger displays. Much effort has been made on making this interaction more natural and more intuitive for the user. The use of computer vision for this purpose has been well researched as it provides freedom and mobility to the user and allows them to interact at a distance. Interaction that relies on monocular computer vision, however, has not been well researched, particularly when used for depth information recovery. This thesis aims to investigate the feasibility of using monocular computer vision to allow bare-hand interaction with large display systems from a distance. By taking into account the location of the user and the interaction area available, a dynamic virtual touchscreen can be estimated between the display and the user. In the process, theories and techniques that make interaction with computer display as easy as pointing to real world objects is explored. Studies were conducted to investigate the way human point at objects naturally with their hand and to examine the inadequacy in existing pointing systems. Models that underpin the pointing strategy used in many of the previous interactive systems were formalized. A proof-of-concept prototype is built and evaluated from various user studies. Results from this thesis suggested that it is possible to allow natural user interaction with large displays using low-cost monocular computer vision. Furthermore, models developed and lessons learnt in this research can assist designers to develop more accurate and natural interactive systems that make use of human’s natural pointing behaviours.