Desktops, laptops, smartphones, tablets, and the Kinect, oh my! With so many devices available to the average consumer, the limitations and pitfalls of each interface are becoming more apparent. Swimming in devices, users often have to stop and think about how to interact with each device to accomplish the current tasks at hand. The goal of this thesis is to minimize user cognitive effort in handling multiple devices by creating a context aware hybrid interface. The context aware system will be explored through the hybridization of gesture and touch interfaces using a multi-touch coffee table and the next-generation Microsoft Kinect. Coupling gesture and touch interfaces creates a novel multimodal interface that can leverage the benefits of both gestures and touch. The hybrid interface is able to utilize the more intuitive and dynamic use of gestures, while maintaining the precision of a tactile touch interface. Joining these two interfaces in an intuitive and context aware way will open up a new avenue for design and innovation.

Digital painting is the field of software designed to provide artists a virtual medium to emulate the experience and results of physical drawing. Several hardware and software components come together to form a whole workflow, ranging from the physical input devices, to the stroking process, to the texture content authorship. This thesis explores an artist-friendly approach to synthesize the textures that give life to digital brush strokes. Most painting software provides a limited library of predefined brush textures. They aim to offer styles approximating physical media like paintbrushes, pencils, markers, and airbrushes. Often these are static bitmap textures that are stamped onto the canvas at repeating intervals, causing discernible repetition artifacts. When more variety is desired, artists often download commercially available brush packs that expand the library of styles. However, included and supplemental brush packs are not easily artist-customizable. In recent years, a separate field of digital art tooling has seen the popular growth of node-based procedural content generation. 3D models, shaders, and materials are commonly authored by artists using functions that can be linked together in a visual programming environment called a node graph. In this work, the feasibility is tested of using a node graph to procedurally generate highly customizable brush textures. The system synthesizes textures that adapt to parameters like pen pressure and stretch along the full length of each brush stroke instead of stamping repetitively. The result is a more flexible and artist-friendly way to define, share, and tweak brush textures used in digital painting.

The term e-Government refers to providing citizens a series of services that can be conveniently conducted over the Internet. However, the potential to redefine and transform e-Government increasingly relies on citizens successfully establishing and managing a user account profile online. E-Government has not adequately addressed user-centric designs for social inclusion of all citizens on e-Government websites. There is a lack of research on the usability of user account management, and a clear lack of innovation in incorporating user-friendly authentication interfaces to accommodate a diverse user population given the wealth of existing research in web authentication techniques within Identity Management. The problem is e-Government has no standardized approach to evaluate and compare the usability of user account interfaces to accommodate a diverse user population and encourage improvements in making user account interfaces more user-friendly and accessible to citizens online. This study proposed extending a well-established usability evaluation methodology called GOMS to evaluate e-Government security interfaces for usability. GOMS, which comprises of Goals, Operations, Methods, and Selection, was used to compare the task time users took to complete similar goals on different websites. GOMS was extended to include Security Cases, which are security related goals users desire to accomplish along with the selected link and trail necessary to satisfy those goals. An observational study was conducted to capture the task time 31 users took to complete similar Security Cases on three popular e-Government websites (DMV.CA.gov, HealthCare.gov, and USPS.com). The study initially defined a catalog of six Security Cases specific to user account management and then established benchmark time predictions for each of the Security Cases using CogTool. The six Security Cases selected were as follows: Registration, Login, Change Settings, Forgot Password, Change Password, and Logout. The task time to complete each of the six Security Case on the three websites, along with statistical analysis and CogTool’s benchmark time predications, were used to quantify and compare the usability of these three websites. In order to capture demographic data and assess participant’s satisfaction using the website, the study conducted a post evaluation survey using the System Usability Scale (SUS). The survey captured age, gender, education, user satisfaction, and computer/security knowledge for each participant to assess design considerations to accommodate a diverse population. Finally, a library of Security Cases was established to compare and highlight the more effective user account interface designs on the three selected e-Government websites. This study found task time data from similar Security Cases could be categorized and used to successfully compare and highlight more effective user account interface designs. The study revealed gender and education had no distinctions in task time when performing user account management related tasks. The study also revealed seniors took significantly longer than any other age group to complete complex user account management interfaces. Additionally, CogTool did not prove to be effective in establishing reliable task time predictions to establish as benchmarks. The study concluded the GOMS method could successfully be used to establish a set of task time metrics in a catalog of Security Cases that can be used to evaluate and compare the usability of user account interfaces to accommodate a diverse user population on e-Government websites. Future usability research should be conducted to evaluate if there is a performance relationship between age and security interface complexity. Future research should also further evaluate GOMS as a viable methodology to evaluate other security interfaces not limited to e-Government and expand upon the library of Security Cases to highlight effective security interfaces designs on other websites to accommodate a diverse user population.

Mobile technologies present an opportunity for scholars and practitioners to extend the application of instructional design theories and models to a mobile learning environment. The goal was to examine mobile learning design and development issues, validate and extend the instructional design theory, Component Display Theory (CDT), to the development of mobile learning activities, and recommend guiding principles for mobile learning system development. Using a formative research approach, which focuses on improving design theory for instructional practices and processes, CDT was used to design a tutorial mobile application targeting faculty professional development. This design instance was formatively evaluated to determine how CDT can be used to guide the design and development of a mobile learning environment; the key processes that are pertinent to translating instructional design plans into mobile learning lessons; and the challenges and issues in designing instruction for a mobile learning environment. The findings resulted in the identification of variables and factors related to the instructional strategies, design variables, and the learning system that affected the application of the CDT. Recommendations and further research opportunities are presented to increase practitioner use of the theory and to address learner and organizational readiness. This research contributes to the field of instructional design and development by examining how underlying theories, principles, and frameworks can be applied to the design and development of mobile learning systems.

Diabetes is the seventh leading cause of death in the United States. With the population rapidly aging, it is expected that 1 out of 3 Americans will have diabetes by 2050. Mobile devices and mobile applications have the potential to contribute to diabetes self-care by allowing users to manage their diabetes by keeping track of their blood glucose levels. Usability is important for systems that help people self-manage conditions such as diabetes. Age and diabetes-related cognitive decline might intensify the impact of usability issues for the users who need these mobile applications the most. As highlighted by usability researchers, the context of use (i.e. environment, user, task, and technology) has a significant impact on usability. The environment (lighting, temperature, audio and visual distractions, etc.) is of special interest to the mobile usability arena since in the case of mobile devices, is always changing. This dissertation aims to support the claim that context and more specifically environmental distraction such as semantic auditory distractions impact the usability of mobile applications. In doing so, it attempts to answer the following research questions: 1) Does semantic auditory distractions reduce the effectiveness of a blood glucose tracking mobile application? 2) Does semantic auditory distractions reduce the efficiency of a blood glucose tracking mobile application? 3) Does semantic auditory distractions reduce the user satisfaction of a blood glucose tracking mobile application? To answer the study research questions, a true experimental design was performed involving 30 adults with type 2 diabetes. Participants were paired based on their age and experience with smartphones and randomly assigned to the control (no semantic auditory distractions) or experimental (semantic auditory distractions) group. Research questions were tested using the general linear model. The results of this study confirmed that semantic auditory distractions have a significant effect on efficiency and effectiveness, and hence they need to be taken into account when evaluating mobile usability. This study also showed that semantic auditory distractions have no significant effect on user satisfaction. This dissertation enhances the current knowledge about the impact of semantic auditory distractions on the usability of mobile applications within the diabetic senior population.

Traditional archival media such as paper, film, photographs, etc. contain a vast storage of knowledge. Much of this knowledge is applicable to current business and scientific problems, and offers solutions; consequently, there is value in extracting this information. While it is possible to manually extract the content, this technique is not feasible for large knowledge repositories due to cost and time. In this thesis, we develop a system that can extract such knowledge automatically from large repositories. A Graphical User Interface that permits users to indicate the location of the knowledge components (indexes) is developed, and software features that permit automatic extraction of indexes from similar documents is presented. The indexes and the documents are stored in a persistentdata store.The system is tested on a University Registrar’s legacy paper-based transcript repository. The study shows that the system provides a good solution for large-scale extraction of knowledge from archived paper and other media.

From object tracking to 3D reconstruction, RGB-Depth (RGB-D) camera networks play an increasingly important role in many vision and graphics applications. With the recent explosive growth of Augmented Reality (AR) and Virtual Reality (VR) platforms, utilizing camera RGB-D camera networks to capture and render dynamic physical space can enhance immersive experiences for users. To maximize coverage and minimize costs, practical applications often use a small number of RGB-D cameras and sparsely place them around the environment for data capturing. While sparse color camera networks have been studied for decades, the problems of extrinsic calibration of and rendering with sparse RGB-D camera networks are less well understood. Extrinsic calibration is difficult because of inappropriate RGB-D camera models and lack of shared scene features. Due to the significant camera noise and sparse coverage of the scene, the quality of rendering 3D point clouds is much lower compared with synthetic models. Adding virtual objects whose rendering depend on the physical environment such as those with reflective surfaces further complicate the rendering pipeline. In this dissertation, I propose novel solutions to tackle these challenges faced by RGB-D camera systems. First, I propose a novel extrinsic calibration algorithm that can accurately and rapidly calibrate the geometric relationships across an arbitrary number of RGB-D cameras on a network. Second, I propose a novel rendering pipeline that can capture and render, in real-time, dynamic scenes in the presence of arbitrary-shaped reflective virtual objects. Third, I have demonstrated a teleportation application that uses the proposed system to merge two geographically separated 3D captured scenes into the same reconstructed environment. To provide a fast and robust calibration for a sparse RGB-D camera network, first, the correspondences between different camera views are established by using a spherical calibration object. We show that this approach outperforms other techniques based on planar calibration objects. Second, instead of modeling camera extrinsic using rigid transformation that is optimal only for pinhole cameras, different view transformation functions including rigid transformation, polynomial transformation, and manifold regression are systematically tested to determine the most robust mapping that generalizes well to unseen data. Third, the celebrated bundle adjustment procedure is reformulated to minimize the global 3D projection error so as to fine-tune the initial estimates. To achieve a realistic mirror rendering, a robust eye detector is used to identify the viewer's 3D location and render the reflective scene accordingly. The limited field of view obtained from a single camera is overcome by our calibrated RGB-D camera network system that is scalable to capture an arbitrarily large environment. The rendering is accomplished by raytracing light rays from the viewpoint to the scene reflected by the virtual curved surface. To the best of our knowledge, the proposed system is the first to render reflective dynamic scenes from real 3D data in large environments. Our scalable client-server architecture is computationally efficient - the calibration of a camera network system, including data capture, can be done in minutes using only commodity PCs.

Computers have been assisting in recording, sound synthesis and other fields of music production for quite some time. The actual performance of music continues to be an area in which human players are chosen over computer performers. Musical performance is an area in which personalization is more important than consistency. Human players play with each other, reacting to phrases and ideas created by the players that they are playing with. Computer performers lack the ability to react to the changes in the performance that humans perceive naturally, giving the human players an advantage over the computer performers. This thesis creates a framework for describing unique musical performers that can play along in realtime with human players. FrTime, a reactive programming language, is used to constantly create new musical phrases. Musical phrases are constructed by unique user programmed performers and by chord changes that the framework provides. The reactive language creates multiple musical phrases for each point in time. A simple module which chooses musical phrases to be performed at the time of performance is created.

Playing chess on a smart phone has gained popularity in the last few years, offering the convenience of correspondence play, automatic recording of a game, etc. Although a good number of players love playing chess on a tablet/smart phone, it doesn't come close to the experience of playing over the traditional board. The feel and pleasure are more real when playing face down with the opponent sitting across each other rather than playing in mobile devices. This is especially true during chess tournaments. It would be ideal to enhance the experience of playing chess on board with the features of chess playing on smart phones. Based on the design of a roll able smart chess board, an android app has been implemented to interact with the board. It reads signals from the smart chess board and maps the movements of the chess pieces to the phone. The recorded play would be used as input for game analysis. The design and implementation of a server for playing and reviewing a game online have also been studied in this thesis.

This dissertation addresses the difficulties of semantic segmentation when dealing with an extensive collection of images and 3D point clouds. Due to the ubiquity of digital cameras that help capture the world around us, as well as the advanced scanning techniques that are able to record 3D replicas of real cities, the sheer amount of visual data available presents many opportunities for both academic research and industrial applications. But the mere quantity of data also poses a tremendous challenge. In particular, the problem of distilling useful information from such a large repository of visual data has attracted ongoing interests in the fields of computer vision and data mining. Structural Semantics are fundamental to understanding both natural and man-made objects. Buildings, for example, are like languages in that they are made up of repeated structures or patterns that can be captured in images. In order to find these recurring patterns in images, I present an unsupervised frequent visual pattern mining approach that goes beyond co-location to identify spatially coherent visual patterns, regardless of their shape, size, locations and orientation. First, my approach categorizes visual items from scale-invariant image primitives with similar appearance using a suite of polynomial-time algorithms that have been designed to identify consistent structural associations among visual items, representing frequent visual patterns. After detecting repetitive image patterns, I use unsupervised and automatic segmentation of the identified patterns to generate more semantically meaningful representations. The underlying assumption is that pixels capturing the same portion of image patterns are visually consistent, while pixels that come from different backdrops are usually inconsistent. I further extend this approach to perform automatic segmentation of foreground objects from an Internet photo collection of landmark locations. New scanning technologies have successfully advanced the digital acquisition of large-scale urban landscapes. In addressing semantic segmentation and reconstruction of this data using LiDAR point clouds and geo-registered images of large-scale residential areas, I develop a complete system that simultaneously uses classification and segmentation methods to first identify different object categories and then apply category-specific reconstruction techniques to create visually pleasing and complete scene models.