Multimodal interfaces are becoming increasingly important with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. This thesis investigates systems support for web browsing in a multimodal interface. Specifically, we outline the design and implementation of a software framework that integrates hyperlink and voice interaction. This enables the user to engage in out-of-turn interactions to personalize access at an information site. For the developer, the framework enables the creation of sites that adapt to the needs of users, yet permits fine-grained control over what interactions to support. Design methodology, implementation details, and two case studies are presented. / Master of Science

The rise of ubiquitous computing devices has provided the catalyst for the next generation World Wide Web, one that shifts the focus from the desktop computer to mobile devices such as cell phones and PDAs, in an ever increasing range of modalities. Web interaction management in this setting must contend with a plethora of interaction interfaces and a diverse range of content types in addition to helping realize the full potential of multimodality (i.e., supporting flexible and personalized interactions between humans and sites). This thesis presents WS://IM, a new software framework for web interaction management that is capable of supporting multimodal interactions. In addition to presenting a loosely bundled, factorized architecture that supports hyperlink interaction, WS://IM has the unique facilitation for out-of-turn interaction. Out-of-turn interaction is a novel technique that helps realize mixed-initiative interactions between humans and Web sites. Design methodology, implementation details, and exposition through three implemented case studies are provided. / Master of Science

Personalization constitutes the mechanisms and technologies necessary to customize information access to the end-user. It can be defined as the automatic adjustment of information content, structure, and presentation. The central thesis of this dissertation is that modeling interaction explicitly in a representation, and studying how partial information can be harnessed in it by program transformations to direct the flow of the interaction, can provide insight into, reveal opportunities for, and define a model for personalized interaction. To evaluate this thesis, a formal modeling methodology is developed for personalizing interactions with information systems, especially hierarchical hypermedia, based on program transformations. The predominant form of personalized interaction developed in this thesis is out-of-turn interaction, a technique which empowers the user to take the initiative in a user--system dialog by providing unsolicited, but relevant, information out-of-turn. Out-of-turn interaction helps flexibly bridge any mismatch between the user's model of information seeking and the system's hardwired hyperlink structure in a manner fundamentally different from extant solutions, such as multiple faceted browsing classifications and search tools. This capability is showcased through two interaction interfaces using alternate modalities to capture and communicate out-of-turn information to the underlying system: a toolbar embedded into a traditional browser for out-of-turn textual input and voice-enabled content pages for out-of-turn speech input. The specific research issues addressed involve identifying and developing representations and transformations suitable for general classes of hierarchical hypermedia, providing supplemental interactions for improving the personalized experience, and studying user's (out-of-turn) interactions with resulting systems. / Ph. D.