Doctor of Philosophy(PhD) / Digital photography has not only changed the nature of photography and the photographic process, but also the manner in which we share photographs and tell stories about them. Some traditional methods, such as the family photo album or passing around piles of recently developed snapshots, are lost to us without requiring the digital photos to be printed. The current, purely digital, methods of sharing do not provide the same experience as printed photographs, and they do not provide effective face-to-face social interaction around photographs, as experienced during storytelling. Research has found that people are often dissatisfied with sharing photographs in digital form. The recent emergence of the tabletop interface as a viable multi-user direct-touch interactive large horizontal display has provided the hardware that has the potential to improve our collocated activities such as digital photograph sharing. However, while some software to communicate with various tabletop hardware technologies exists, software aspects of tabletop user interfaces are still at an early stage and require careful consideration in order to provide an effective, multi-user immersive interface that arbitrates the social interaction between users, without the necessary computer-human interaction interfering with the social dialogue. This thesis presents PhoTable, a social interface allowing people to effectively share, and tell stories about, recently taken, unsorted digital photographs around an interactive tabletop. In addition, the computer-arbitrated digital interaction allows PhoTable to capture the stories told, and associate them as audio metadata to the appropriate photographs. By leveraging the tabletop interface and providing a highly usable and natural interaction we can enable users to become immersed in their social interaction, telling stories about their photographs, and allow the computer interaction to occur as a side-effect of the social interaction. Correlating the computer interaction with the corresponding audio allows PhoTable to annotate an automatically created digital photo album with audible stories, which may then be archived. These stories remain useful for future sharing -- both collocated sharing and remote (e.g. via the Internet) -- and also provide a personal memento both of the event depicted in the photograph (e.g. as a reminder) and of the enjoyable photo sharing experience at the tabletop. To provide the necessary software to realise an interface such as PhoTable, this thesis explored the development of Cruiser: an efficient, extensible and reusable software framework for developing tabletop applications. Cruiser contributes a set of programming libraries and the necessary application framework to facilitate the rapid and highly flexible development of new tabletop applications. It uses a plugin architecture that encourages code reuse, stability and easy experimentation, and leverages the dedicated computer graphics hardware and multi-core processors of modern consumer-level systems to provide a responsive and immersive interactive tabletop user interface that is agnostic to the tabletop hardware and operating platform, using efficient, native cross-platform code. Cruiser's flexibility has allowed a variety of novel interactive tabletop applications to be explored by other researchers using the framework, in addition to PhoTable. To evaluate Cruiser and PhoTable, this thesis follows recommended practices for systems evaluation. The design rationale is framed within the above scenario and vision which we explore further, and the resulting design is critically analysed based on user studies, heuristic evaluation and a reflection on how it evolved over time. The effectiveness of Cruiser was evaluated in terms of its ability to realise PhoTable, use of it by others to explore many new tabletop applications, and an analysis of performance and resource usage. Usability, learnability and effectiveness of PhoTable was assessed on three levels: careful usability evaluations of elements of the interface; informal observations of usability when Cruiser was available to the public in several exhibitions and demonstrations; and a final evaluation of PhoTable in use for storytelling, where this had the side effect of creating a digital photo album, consisting of the photographs users interacted with on the table and associated audio annotations which PhoTable automatically extracted from the interaction. We conclude that our approach to design has resulted in an effective framework for creating new tabletop interfaces. The parallel goal of exploring the potential for tabletop interaction as a new way to share digital photographs was realised in PhoTable. It is able to support the envisaged goal of an effective interface for telling stories about one's photos. As a serendipitous side-effect, PhoTable was effective in the automatic capture of the stories about individual photographs for future reminiscence and sharing. This work provides foundations for future work in creating new ways to interact at a tabletop and to the ways to capture personal stories around digital photographs for sharing and long-term preservation.

Deictic gestures, which often manifest as pointing, are an important part of interpersonal communication over shared artifacts on surfaces, such as a map on a table. However, in computer-supported distributed settings, deictic gestures can be difficult to see and understand. This problem can be solved through visualizing hands and arms above distributed surfaces, but current solutions are computationally and programmatically expensive, rely on a limited understanding of how gestures are executed and used, and remain largely unevaluated with regards to their effectiveness. This dissertation describes a solution to these problems in four parts: 1. Qualitative observational studies, both laboratory-based and in the wild, that lead to a greater understanding of how gestures are made over surfaces and what parts of a gesture are important to represent. In particular, these observations identified the height of a gesture as a characteristic not well-supported in distributed groupware. 2. A description of the design space available for representing gestures and candidate designs for showing the height of a gesture in distributed groupware. 3. Experimental evaluations of embodiments that include the representation of gesture height. 4. A toolkit for facilitating the capture and representation of gestures in distributed groupware. This work is the first to describe how deictic gestures are made over surfaces and how to visualize these gestures in distributed settings. The KinectArms Toolkit is the first toolkit to allow developers to add rich arm and hand representations to groupware without undue cost or development effort. This work is important because it provides researchers, designers, and developers with new tools for understanding and supporting communication in distributed settings.

Doctor of Philosophy(PhD) / Digital photography has not only changed the nature of photography and the photographic process, but also the manner in which we share photographs and tell stories about them. Some traditional methods, such as the family photo album or passing around piles of recently developed snapshots, are lost to us without requiring the digital photos to be printed. The current, purely digital, methods of sharing do not provide the same experience as printed photographs, and they do not provide effective face-to-face social interaction around photographs, as experienced during storytelling. Research has found that people are often dissatisfied with sharing photographs in digital form. The recent emergence of the tabletop interface as a viable multi-user direct-touch interactive large horizontal display has provided the hardware that has the potential to improve our collocated activities such as digital photograph sharing. However, while some software to communicate with various tabletop hardware technologies exists, software aspects of tabletop user interfaces are still at an early stage and require careful consideration in order to provide an effective, multi-user immersive interface that arbitrates the social interaction between users, without the necessary computer-human interaction interfering with the social dialogue. This thesis presents PhoTable, a social interface allowing people to effectively share, and tell stories about, recently taken, unsorted digital photographs around an interactive tabletop. In addition, the computer-arbitrated digital interaction allows PhoTable to capture the stories told, and associate them as audio metadata to the appropriate photographs. By leveraging the tabletop interface and providing a highly usable and natural interaction we can enable users to become immersed in their social interaction, telling stories about their photographs, and allow the computer interaction to occur as a side-effect of the social interaction. Correlating the computer interaction with the corresponding audio allows PhoTable to annotate an automatically created digital photo album with audible stories, which may then be archived. These stories remain useful for future sharing -- both collocated sharing and remote (e.g. via the Internet) -- and also provide a personal memento both of the event depicted in the photograph (e.g. as a reminder) and of the enjoyable photo sharing experience at the tabletop. To provide the necessary software to realise an interface such as PhoTable, this thesis explored the development of Cruiser: an efficient, extensible and reusable software framework for developing tabletop applications. Cruiser contributes a set of programming libraries and the necessary application framework to facilitate the rapid and highly flexible development of new tabletop applications. It uses a plugin architecture that encourages code reuse, stability and easy experimentation, and leverages the dedicated computer graphics hardware and multi-core processors of modern consumer-level systems to provide a responsive and immersive interactive tabletop user interface that is agnostic to the tabletop hardware and operating platform, using efficient, native cross-platform code. Cruiser's flexibility has allowed a variety of novel interactive tabletop applications to be explored by other researchers using the framework, in addition to PhoTable. To evaluate Cruiser and PhoTable, this thesis follows recommended practices for systems evaluation. The design rationale is framed within the above scenario and vision which we explore further, and the resulting design is critically analysed based on user studies, heuristic evaluation and a reflection on how it evolved over time. The effectiveness of Cruiser was evaluated in terms of its ability to realise PhoTable, use of it by others to explore many new tabletop applications, and an analysis of performance and resource usage. Usability, learnability and effectiveness of PhoTable was assessed on three levels: careful usability evaluations of elements of the interface; informal observations of usability when Cruiser was available to the public in several exhibitions and demonstrations; and a final evaluation of PhoTable in use for storytelling, where this had the side effect of creating a digital photo album, consisting of the photographs users interacted with on the table and associated audio annotations which PhoTable automatically extracted from the interaction. We conclude that our approach to design has resulted in an effective framework for creating new tabletop interfaces. The parallel goal of exploring the potential for tabletop interaction as a new way to share digital photographs was realised in PhoTable. It is able to support the envisaged goal of an effective interface for telling stories about one's photos. As a serendipitous side-effect, PhoTable was effective in the automatic capture of the stories about individual photographs for future reminiscence and sharing. This work provides foundations for future work in creating new ways to interact at a tabletop and to the ways to capture personal stories around digital photographs for sharing and long-term preservation.

The Design of Table-Centric Interactive Spaces, by Daniel J. Wigdor A thesis submitted in partial conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Computer Science, University of Toronto © Copyright by Daniel J. Wigdor, 2008 Direct-touch tabletops offer compelling uses as direct, multi-touch, multi-user displays for face to face collaborative work. As task complexity and group size increase, the addition to the tabletop of multiple, vertical displays allows for more information content, while reducing the need to multiplex the tabletop display area. We dub such systems table-centric interactive spaces. Although compelling, these spaces offer unique challenges. In particular, the displays in such spaces are seen by the users at angles not typically found in combination in other environments. First, the viewing imagery shown on a horizontal display by seated participants means that that imagery is distorted, receding away from the users’ eyes. Second, the sharing of information by users sitting around a horizontal display necessitates that on-screen content be oriented at non-optimal angles for some subset of those users. Third, positioning vertical displays around the table means that some subset of the seated users will be looking at those displays at odd angles. In this thesis, we investigate the challenges associated with these viewing angles. We begin with a examination of related work, including tabletop technology and interaction techniques. Next, we report the results of controlled experiments measuring performance of reading, graphical perception, and ancillary display control under the angles we identified. Next, we present a set of design issues encountered in our work with table-centric spaces. We then review a series of interaction techniques built to address those issues. These techniques are evaluated through the construction and validation of an application scenario. Through these examinations, we hope to provide designers with insights as to how to enable users to take full advantage of ancillary displays, while maintaining the advantages and affordances of collocated table-centric work.

The Design of Table-Centric Interactive Spaces, by Daniel J. Wigdor A thesis submitted in partial conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Computer Science, University of Toronto © Copyright by Daniel J. Wigdor, 2008 Direct-touch tabletops offer compelling uses as direct, multi-touch, multi-user displays for face to face collaborative work. As task complexity and group size increase, the addition to the tabletop of multiple, vertical displays allows for more information content, while reducing the need to multiplex the tabletop display area. We dub such systems table-centric interactive spaces. Although compelling, these spaces offer unique challenges. In particular, the displays in such spaces are seen by the users at angles not typically found in combination in other environments. First, the viewing imagery shown on a horizontal display by seated participants means that that imagery is distorted, receding away from the users’ eyes. Second, the sharing of information by users sitting around a horizontal display necessitates that on-screen content be oriented at non-optimal angles for some subset of those users. Third, positioning vertical displays around the table means that some subset of the seated users will be looking at those displays at odd angles. In this thesis, we investigate the challenges associated with these viewing angles. We begin with a examination of related work, including tabletop technology and interaction techniques. Next, we report the results of controlled experiments measuring performance of reading, graphical perception, and ancillary display control under the angles we identified. Next, we present a set of design issues encountered in our work with table-centric spaces. We then review a series of interaction techniques built to address those issues. These techniques are evaluated through the construction and validation of an application scenario. Through these examinations, we hope to provide designers with insights as to how to enable users to take full advantage of ancillary displays, while maintaining the advantages and affordances of collocated table-centric work.

Above-surface interaction is a new and exciting topic in the field of human-computer interaction (HCI). It focuses on the design and evaluation of systems that humans can operate by moving their hands in the space above or in front of interactive displays. While many technologies emerge that make such systems possible, much research is still needed to make this interaction as natural and effortless as possible. First this thesis presents a set of guidelines for designing above-surface interactions, a collection of widgets that were designed based on these guidelines, and a system that can approximate the height of hands above a diffused surface illumination (DSI) device without any additional sensors. Then the thesis focuses on interaction techniques for activating graphical widgets located in this above-surface space. Finally, it presents a pair of studies that were conducted to investigate item selection in the space above a multitouch surface. The first study was conducted to elicit a set of gestures for above-table widget activation from a group of users. Several gestures were proposed by the designers to be compared with the user-generated gestures. The follow-up study was conducted to evaluate and compare these gestures based on their performance. The findings of these studies showed that there was no clear agreement on what gestures should be used to select objects in mid-air, and that performance was better when using gestures that were chosen less frequently, but predicted to be better by the designers, as opposed to those most frequently suggested by participants.

Above-surface interaction is a new and exciting topic in the field of human-computer interaction (HCI). It focuses on the design and evaluation of systems that humans can operate by moving their hands in the space above or in front of interactive displays. While many technologies emerge that make such systems possible, much research is still needed to make this interaction as natural and effortless as possible. First this thesis presents a set of guidelines for designing above-surface interactions, a collection of widgets that were designed based on these guidelines, and a system that can approximate the height of hands above a diffused surface illumination (DSI) device without any additional sensors. Then the thesis focuses on interaction techniques for activating graphical widgets located in this above-surface space. Finally, it presents a pair of studies that were conducted to investigate item selection in the space above a multitouch surface. The first study was conducted to elicit a set of gestures for above-table widget activation from a group of users. Several gestures were proposed by the designers to be compared with the user-generated gestures. The follow-up study was conducted to evaluate and compare these gestures based on their performance. The findings of these studies showed that there was no clear agreement on what gestures should be used to select objects in mid-air, and that performance was better when using gestures that were chosen less frequently, but predicted to be better by the designers, as opposed to those most frequently suggested by participants.