This dissertation addresses the difficulties of semantic segmentation when dealing with an extensive collection of images and 3D point clouds. Due to the ubiquity of digital cameras that help capture the world around us, as well as the advanced scanning techniques that are able to record 3D replicas of real cities, the sheer amount of visual data available presents many opportunities for both academic research and industrial applications. But the mere quantity of data also poses a tremendous challenge. In particular, the problem of distilling useful information from such a large repository of visual data has attracted ongoing interests in the fields of computer vision and data mining. Structural Semantics are fundamental to understanding both natural and man-made objects. Buildings, for example, are like languages in that they are made up of repeated structures or patterns that can be captured in images. In order to find these recurring patterns in images, I present an unsupervised frequent visual pattern mining approach that goes beyond co-location to identify spatially coherent visual patterns, regardless of their shape, size, locations and orientation. First, my approach categorizes visual items from scale-invariant image primitives with similar appearance using a suite of polynomial-time algorithms that have been designed to identify consistent structural associations among visual items, representing frequent visual patterns. After detecting repetitive image patterns, I use unsupervised and automatic segmentation of the identified patterns to generate more semantically meaningful representations. The underlying assumption is that pixels capturing the same portion of image patterns are visually consistent, while pixels that come from different backdrops are usually inconsistent. I further extend this approach to perform automatic segmentation of foreground objects from an Internet photo collection of landmark locations. New scanning technologies have successfully advanced the digital acquisition of large-scale urban landscapes. In addressing semantic segmentation and reconstruction of this data using LiDAR point clouds and geo-registered images of large-scale residential areas, I develop a complete system that simultaneously uses classification and segmentation methods to first identify different object categories and then apply category-specific reconstruction techniques to create visually pleasing and complete scene models.

Simultaneous localization and mapping is the technique to construct a 3D map of unknown environment. With the increasing popularity of RGB-depth (RGB-D) sensors such as the Microsoft Kinect, there have been much research on capturing and reconstructing 3D environments using a movable RGB-D sensor. The key process behind these kinds of simultaneous location and mapping (SLAM) systems is the iterative closest point or ICP algorithm, which is an iterative algorithm that can estimate the rigid movement of the camera based on the captured 3D point clouds. While ICP is a well-studied algorithm, it is problematic when it is used in scanning large planar regions such as wall surfaces in a room. The lack of depth variations on planar surfaces makes the global alignment an ill-conditioned problem. In this thesis, we present a novel approach for registering 3D point clouds by combining both color and depth information. Instead of directly searching for point correspondences among 3D data, the proposed method first extracts features from the RGB images, and then back-projects the features to the 3D space to identify more reliable correspondences. These color correspondences form the initial input to the ICP procedure which then proceeds to refine the alignment. Experimental results show that our proposed approach can achieve better accuracy than existing SLAMs in reconstructing indoor environments with large planar surfaces.

Many software development projects fail because they do not meet the needs of users, are over-budget, and abandoned. To address this problem, the user requirements elicitation process was modified based on principles of Appreciative Inquiry. Appreciative Inquiry, commonly used in organizational development, aims to build organizations, processes, or systems based on success stories using a hopeful vision for an ideal future. Spanning five studies, Appreciative Inquiry was evaluated for its effectiveness with eliciting user requirements. In the first two cases, it was compared with traditional approaches with end-users and proxy-users. The third study was a quasi-experiment comparing the use of Appreciative Inquiry in different phases of in the software development cycle. The final two case studies combined all lessons learned using Appreciative Inquiry, with multiple case studies to gain additional understanding for the requirements gathered during various project phases. Each study evaluated the requirements gathered, developer and user attitudes, and the Appreciative Inquiry process itself. Requirements were evaluated for the quantity and their type regardless of whether they were implemented or not. Attitudes were evaluated for process feedback, as well as requirements and project commitment. The Appreciative Inquiry process was evaluated with differing groups, projects, and project phases to determine how and when it is best applied. Potentially interceding factors were also evaluated including: team effectiveness, emotional intelligence, perceived stress, the experience of the facilitator, and the development project type itself. Appreciative Inquiry produced positive results for the participants, the requirements obtained, and the general requirements eliciting-process. Appreciative Inquiry demonstrated benefits to the requirements gathered by increasing the number of unique requirements as well as identifying more quality-based (non-functional) and forward-looking requirements. It worked well with defined projects, when there was time for participants to reflect on the thought-provoking questions, structured questions and extra time to facilitate the extraction and translation of requirements, and a knowledgeable interviewer. The participants (end-users and developers) expressed improved vision and confidence. End-users participated consistently with immediate buy-in and enthusiasm, especially those users who were technically-inhibited. Development teams expressed improved confidence, and improved user communication and understanding.

Today, advances in wireless sensor networks are making it possible to capture large amounts of information about a person and their interaction within their home environment. However, what is missing is how to ensure the security of the collected data and its use to alter human behavior for positive benefit. In this research, exploration was conducted involving the "infrastructure" and "intelligence" aspects of a wireless sensor network through a Behavior Modification Sensor System. First was to understand how a secure wireless sensor network could be established through the symmetric distribution of keys (the securing of the infrastructure), and it involves the mathematical analysis of a novel key pre-distribution scheme. Second explores via field testing the "intelligence" level of the system. This was meant to support the generation of persuasive messages built from the integration of a person's physiological and living pattern data in persuading physical activity behavior change associated with daily walking steps. This system was used by an elderly female in a three-month study. Findings regarding the "infrastructure" or the novel key pre-distribution scheme in comparison to three popular key distribution methods indicates that it offers greater network resiliency to security threats (i.e., 1/2^32 times lower), better memory utilization (i.e., 53.9% less), but higher energy consumption (i.e., 2% higher) than its comparison group. Findings from the "intelligence" level of the research posit that using a person's physiological and living pattern data may allow for more "information rich" and stronger persuasive messages. Findings indicate that the study participant was able to change and improve her average daily walking steps by 61% over a pre-treatment period. As the study participant increased her physical activity, changes in her living pattern were also observed (e.g., time spent watching television decreased while time spent engaged in walking increased by an average of 15 minutes per day). Reinforcement of these findings were noted between a pre and post-study survey that indicated the study participant moved from a contemplation stage of change where physical activity engagement was intended but not acted upon to an action stage of change where physical activity engagement dominated the new behavior.

Selecting one color can be done with relative ease, but this task becomes more difficult with each subsequent color. Colormoo is an online tool aimed at solving this problem. We implement three algorithms for generating color palettes based off of a starting color. Data is collected for each palette that is generated. Our analysis reveals two of the algorithms are preferred, but under different circumstances. Furthermore, we find that users prefer palettes containing colors that are compatible, but not too similar. With refined heuristics, we believe these techniques can be extended and applied beyond the field of graphic design alone.

Eye-tracking devices can tell us where on the screen a person is looking. Researchers frequently analyze eye-tracking data manually, by examining every frame of a visual stimulus used in an eye-tracking experiment so as to match 2D screen-coordinates provided by the eye-tracker to related objects and content within the stimulus. Such task requires significant manual effort and is not feasible for analyzing data collected from many users, long experimental sessions, and heavily interactive and dynamic visual stimuli. In this dissertation, we present a novel analysis method. We would instrument visualizations that have open source code, and leverage real-time information about the layout of the rendered visual content, to automatically relate gaze-samples to visual objects drawn on the screen. Since such visual objects are shown in a visualization stand for data, the method would allow us to necessarily detect data that users focus on or Data of Interest (DOI). This dissertation has two contributions. First, we demonstrated the feasibility of collecting DOI data for real life visualization in a reliable way which is not self-evident. Second, we formalized the process of collecting and interpreting DOI data and test whether the automated DOI detection can lead to research workflows, and insights not possible with traditional, manual approaches.

In an increasingly mobile world, the combination of mobile computing devices, publicly accessible Wi-Fi hotspots, and camera phones pose a significant threat to alphanumeric passwords in public environments. Graphical passwords, introduced as an alternative to alphanumerical passwords, help prevent successful shoulder-surfing attacks – covertly observing or recording a password login session, however, most cannot prevent intersection analysis on the data collected through shoulder-surfing. ColorDots is a new graphical password scheme designed to be easy to use and learn, to prevent successful shoulder-surfing attacks, and to hinder intersection analysis. A software implementation of ColorDots is tested, and the results analyzed. This study showed the ColorDots graphical password scheme does prevent shoulder-surfing, and hinders intersection analysis on digital recordings of multiple shoulder-surfing attacks. Furthermore, ColorDots may be just as convenient to use as alphanumeric passwords, while improving password security in public environments.

This thesis presents the design for a novel haptic interface for large-format touchscreens. Techniques such as electrovibration, ultrasonic vibration, and external braked devices have been developed by other researchers to deliver haptic feedback to touchscreen users. However, these methods do not address the need for spatial constraints that only restrict user motion in the direction of the constraint. This technology gap contributes to the lack of haptic technology available for touchscreen-based upper-limb rehabilitation, despite the prevalent use of haptics in other forms of robotic rehabilitation. The goal of this thesis is to display kinesthetic haptic constraints to the touchscreen user in the form of boundaries and paths, which assist or challenge the user in interacting with the touchscreen. The presented prototype accomplishes this by steering a single wheel in contact with the display while remaining driven by the user. It employs a novel embedded force sensor, which it uses to measure the interaction force between the user and the touchscreen. The haptic response of the device is controlled using this force data to characterize user intent. The prototype can operate in a simulated free mode as well as simulate rigid and compliant obstacles and path constraints. A data architecture has been created to allow the prototype to be used as a peripheral add-on device which reacts to haptic environments created and modified on the touchscreen. The long-term goal of this work is to create a haptic system that enables a touchscreen-based rehabilitation platform for people with upper limb impairments.

In augmented reality, information is often distributed between real and virtual contexts, and often appears at different distances from the viewer. This raises the issues of (1) context switching, when attention is switched between real and virtual contexts, (2) focal distance switching, when the eye accommodates to see information in sharp focus at a new distance, and (3) transient focal blur, when information is seen out of focus, during the time interval of focal distance switching. This dissertation research has quantified the impact of context switching, focal distance switching, and transient focal blur on human performance and eye fatigue in both monocular and binocular viewing conditions. Further, this research has developed a novel font that when seen out-of-focus looks sharper than standard fonts. This SharpView font promises to mitigate the effect of transient focal blur. Developing this font has required (1) mathematically modeling out-of-focus blur with Zernike polynomials, which model focal deficiencies of human vision, (2) developing a focus correction algorithm based on total variation optimization, which corrects out-of-focus blur, and (3) developing a novel algorithm for measuring font sharpness. Finally, this research has validated these fonts through simulation and optical camera-based measurement. This validation has shown that, when seen out of focus, SharpView fonts are as much as 40 to 50% sharper than standard fonts. This promises to improve font legibility in many applications of augmented reality.

Using an explicit integration method in physically based animations has many advantages including conceptual and computational simplicity, however, it re- quires small time steps to ensure low numerical instability. Simulations with large numbers of individually interacting components such as cloth, hair, and fluid models, are limited by the sections of particles most susceptible to error. This results in the need for smaller time steps than required for the majority of the system. These sections can be diverse and dynamic, quickly changing in size and location based on forces in the system. Identifying and handling these trou- blesome sections could allow for a larger time step to be selected, while preventing a breakdown in the simulation. This thesis presents Smarticles (smart particles), a method of individually de- tecting particles exhibiting signs of instability and stabilizing them with minimal adverse effects to visual accuracy. As a result, higher levels of error introduced from large time steps can be tolerated with minimal overhead. Two separate approaches to Smarticles were implemented. They attempt to find oscillating particles by analyzing a particle’s (1) past behavior and (2) behavior with re- spect to its neighbors along a strand. Both versions of Smarticles attempt to correct unstable particles using velocity dampening. Smarticles was applied to a two dimensional hair simulation modeled as a continuum using smooth particle hydrodynamic. Hair strands are formed by linking particles together using one of two methods: position based dynamics or mass-spring forces. Both versions of Smarticles, as well as a control of normal particles, were directly compared and evaluated based on stability and visual fluidity. Hair particles were exposed to various forms of external forces under increasing time step lengths. Testing showed that both versions of Smarticles working together allowed an average increase of 18.62% in the time step length for hair linked with position based dynamics. In addition, Smarticles was able to significantly reduce visible instability at even larger time steps. While these results suggest Smarticles is successful, the method used to correct particle instability may jeopardize other important aspects of the simulation. A more accurate correction method would likely need to be developed to make Smarticles an advantageous method.