User interaction in visual analytic systems is critical to enabling visual data exploration. Through interacting with visualizations, users engage in sensemaking, a process of developing and understanding relationships within datasets through foraging and synthesis. For example, two-dimensional layouts of high-dimensional data can be generated by dimension reduction models, and provide users with an overview of the relationships between information. However, exploring such spatializations can require expertise with the internal mechanisms and parameters of these models. The core contribution of this work is semantic interaction, capable of steering such models without requiring expertise in dimension reduction models, but instead leveraging the domain expertise of the user. Semantic interaction infers the analytical reasoning of the user with model updates, steering the dimension reduction model for visual data exploration. As such, it is an approach to user interaction that leverages interactions designed for synthesis, and couples them with the underlying mathematical model to provide computational support for foraging. As a result, semantic interaction performs incremental model learning to enable synergy between the user's insights and the mathematical model. The contributions of this work are organized by providing a description of the principles of semantic interaction, providing design guidelines through the development of a visual analytic prototype, ForceSPIRE, and the evaluation of the impact of semantic interaction on the analytic process. The positive results of semantic interaction open a fundamentally new design space for designing user interactions in visual analytic systems. This research was funded in part by the National Science Foundation, CCF-0937071 and CCF-0937133, the Institute for Critical Technology and Applied Science at Virginia Tech, and the National Geospatial-Intelligence Agency contract #HMI1582-05-1-2001. / Ph. D.

An essential software development tool, code search engines are expected to provide superior accuracy, usability, and performance. However, prior research has neither (1) summarized, categorized, and compared representative code search engines, nor (2) analyzed the actual expectations that developers have for code search engines. This missing knowledge can empower developers to fully benefit from search engines, academic researchers to uncover promising research directions, and industry practitioners to properly marshal their efforts. This thesis fills the aforementioned gaps by drawing a comprehensive picture of code search engines, including their definition, standard processes, existing solutions, common alternatives, and developers' perspectives. We first study the state of the art in code search engines by analyzing academic papers, industry releases, and open-source projects. We then survey more than a 100 software developers to ascertain their usage of and preferences for code search engines. Finally, we juxtapose the results of our study and survey to synthesize a call-for-action for researchers and industry practitioners to better meet the demands software developers make on code search engines. We present the first comprehensive overview of state-of-the-art code search engines by categorizing and comparing them based on their respective search strategies, applicability, and performance. Our user survey revealed a surprising lack of awareness among many developers w.r.t. code search engines, with a high preference for using general-purpose search engines (e.g., Google) or code repositories (e.g., GitHub) to search for code. Our results also clearly identify typical usage scenarios and sought-after properties of code search engines. Our findings can guide software developers in selecting code search engines most suitable for their programming pursuits, suggest new research directions for researchers, and help programming tool builders in creating effective code search engine solutions. / Master of Science / When developing software, programmers rely on source code search engines to find code snippets related to the programming task at hand. Given their importance for software development, source code engines have become the focus of numerous research and industry projects. However, researchers and developers remain largely unaware of each other's efforts and expectations. As a consequence, developers find themselves struggling to determine which engine would best fit their needs, while researchers remain unaware what developers expect from search engines. This thesis address this problem via a three-pronged approach: (1) it provides a systematic review of the research literature and major engines; (2) it analyzes the results of surveying software developers about their experiences with and expectations for code search engines; (3) it presents actionable insights that can guide future research and industry efforts in code search engines to better meet the needs of software developers.

Automatic Speech Recognition (ASR) research has been primarily focused towards large-scale systems and industry, while other areas that require attention are often over-looked by researchers. For this reason, this research looked at automatic speech recognition at the consumer level. Many individual consumers will purchase and use automatic software recognition for a different purpose than that of the military or commercial industries, such as telecommunications. Consumers who purchase the software for personal use will mainly use ASR for dictation of correspondences and documents. Two ASR dictation software packages were used to conduct the study. The research examined the relationships between (1) speech recognition software training and word accuracy, (2) error-correction time by the user and word accuracy, and (3) correspondence type and word accuracy. The correspondences evaluated were those that resemble Personal, Business, and Technical Correspondences. Word accuracy was assessed after initial system training, five minutes of error-correction time, and ten minutes of error-correction time. Results indicated that word recognition accuracy achieved does affect user satisfaction. It was also found that with increased error-correction time, word accuracy results improved. Additionally, the results found that Personal Correspondence achieved the highest mean word accuracy rate for both systems and that Dragon Systems achieved the highest mean word accuracy recognition for the Correspondences explored in this research. Results were discussed in terms of subjective and objective measures, advantages and disadvantages of speech input, and design recommendations were provided. / Master of Science

Large, high resolution displays combine high pixel density with ample physical dimensions. Combination of these two factors creates a multi-scale workspace where object targeting requires both high speed and high accuracy for nearby and far apart targeting. Modern operating systems support dynamic control-display gain adjustment (i.e. cursor acceleration) that helps to maintain both speed and accuracy. However, very large high resolution displays require broad range of control-display gain ratios. Current interaction techniques attempt to solve the problem by utilizing multiple modes of interaction, where different modes provide different levels of pointer precision. We are investigating the question of the value of allowing users to dynamically choose granularity levels for continuous pointing within single mode of interaction via multiple acceleration curves. Our solution offers different cursor acceleration curves depending on the targeting conditions, thus broadening the range of control-display ratios. Our approach utilizes a consumer multitouch touchpad that allows fast and accurate detection of multiple fingers. A user can choose three different acceleration curves based on how many fingers are used for cursor positioning. Our goal is to investigate the effects of such multi-scale interaction and to compare it against standard single curve interaction. / Master of Science

Although many selection techniques have been proposed and developed over the years, selection by pointing is perhaps the most popular approach for selection. In 3D interfaces, the laser-pointer metaphor is commonly used, since users only have to point to their target from a distance. However, the task of selecting objects that have a small visible area or that are in highly cluttered environments is hard when using pointing techniques. With both indirect and direct pointing techniques in 3D interfaces, smaller targets require higher levels of pointing precision from the user. In addition, issues such as target occlusion as well as hand and tracker jitter negatively affect user performance. Therefore, requiring the user to perform selection in a single precise step may result in users spending more time to select targets so that they can be more accurate (effect known as the speed-accuracy trade-off). We describe an approach to address this issue, called Progressive Refinement. Instead of performing a single precise selection, users gradually reduce the set of selectable objects to reduce the required precision of the task. This approach, however, has an inherent trade-off when compared to immediate selection techniques. Progressive refinement requires a gradual process of selection, often using multiple steps, although each step can be fast, accurate, and nearly effortless. Immediate techniques, on the other hand, involve a single-step selection that requires effort and may be slower and more error-prone. Therefore, the goal of this work was to explore this trade-off. The research includes the design and evaluation of progressive refinement techniques for 3D interfaces, using both pointing- and gesture-based interfaces for single-object selection and volume selection. Our technique designs and other existing selection techniques that can be classified as progressive refinement were used to create a design space. We designed eight progressive refinement techniques and compared them to the most commonly used techniques (for a baseline comparison) and to other state-of-the-art selection techniques in a total of four empirical studies. Based on the results of the studies, we developed a set of design guidelines that will help other researchers design and use progressive refinement techniques. / Ph. D.

Learning from text data often involves a loop of tasks that iterate between foraging for information and synthesizing it in incremental hypotheses. Past research has shown the advantages of using spatial workspaces as a means for synthesizing information through externalizing hypotheses and creating spatial schemas. However, spatializing the entirety of datasets becomes prohibitive as the number of documents available to the analysts grows, particularly when only a small subset are relevant to the tasks at hand. To address this issue, we developed the multi-model semantic interaction (MSI) technique, which leverages user interactions to aid in the display layout (as was seen in previous semantic interaction work), forage for new, relevant documents as implied by the interactions, and then place them in context of the user's existing spatial layout. This results in the ability for the user to conduct both implicit queries and traditional explicit searches. A comparative user study of StarSPIRE discovered that while adding implicit querying did not impact the quality of the foraging, it enabled users to 1) synthesize more information than users with only explicit querying, 2) externalize more hypotheses, 3) complete more synthesis-related semantic interactions. Also, 18% of relevant documents were found by implicitly generated queries when given the option. StarSPIRE has also been integrated with web-based search engines, allowing users to work across vastly different levels of data scale to complete exploratory data analysis tasks (e.g. literature review, investigative journalism). The core contribution of this work is multi-model semantic interaction (MSI) for usable big data analytics. This work has expanded the understanding of how user interactions can be interpreted and mapped to underlying models to steer multiple algorithms simultaneously and at varying levels of data scale. This is represented in an extendable multi-model semantic interaction pipeline. The lessons learned from this dissertation work can be applied to other visual analytics systems, promoting direct manipulation of the data in context of the visualization rather than tweaking algorithmic parameters and creating usable and intuitive interfaces for big data analytics. / Ph. D.

This research focuses on the design of an Extended Reality simulator for training medical professionals in Advanced Trauma Life Support (ATLS) and pulse palpation. Existing pulse simulators have the disadvantages of being bulky, expensive, and unsuitable to be used as training tools. In addition, none of the simulators were designed to incorporate the auditory feedback of the pulse, a crucial component of continuous pulse monitoring. The developed simulator incorporates haptic, visual, and auditory feedback modes. In this work, we also conduct a comparative user study to determine the effect of multimodal feedback on different participants. Participants trained in the Audio-Haptic scenario outperformed those trained in the Haptic only scenario. These values could also be correlated with qualitative user feedback indicating that Audio-Haptic interactions were perceived as superior. With this simulator, we hope to provide medical professionals with an immersive and realistic training tool for learning the skill of palpating pulse. / Master of Science / The medical field demands accurate and precise procedures to be performed by doctors, with no room for error. Traditional training methods consist of the trainer demonstrating the technique and the student duplicating it, which increases the risk of medical errors. Advanced Trauma Life Support (ATLS) is a program designed by the American College of Surgeons to teach physicians a systematic approach to treating trauma patients. Palpating and classifying pulses is one of the steps involved in the ATLS procedure. The majority of existing ATLS and pulse simulators are not fully integrated with haptic and auditory feedback, and there has been very little research on this topic. This work describes the design and development of an Extended Reality ATLS simulator with a pulse simulator for medical student training. We conduct a user study to determine how the Audio-Haptic scenario affects the learnability of palpating pulses and ATLS procedures. Our ATLS simulator aims to provide a comprehensive training module for emergency trauma response practice for medical professionals.

Dark Patterns are a variety of different software designs that are used to manipulate and mislead the users of an application or service. These patterns range from making it harder to end a subscription service, adding additional charges to a purchase, or having the user give out data or personal information. With how widespread and varied dark patterns are, it led to us creating a way to detect and warn users of different dark patterns. In this study, we created Dark Pattern Detector, a Chrome extension that would help users detect and understand three different dark patterns: Hidden Costs, Disguised Ads, and Sneak into Basket. This extension was made to detect each of these patterns on any web page while not requiring any information from the user or their data. Study participants installed the extension and completed a series of tasks given to them that would occur on different websites containing the previous dark patterns. After completing the tasks, the users were surveyed to give feedback on what they thought of the extension and what suggestions for change they had. In the study, we had 40 participants and we found that 50% of the users were completely unfamiliar with dark patterns and that 77.5% have used extensions before. For the five tasks, each one had a majority of the participants successfully complete them. Finally, when asked about what they thought, the majority of the participants gave positive feedback claiming that they found the extension useful, interesting, and a good idea. Many participants also gave useful feedback about what changes or additions they would like to see. With our results, we can help users have a better understanding of dark patterns and have created a baseline for any future research done on dark pattern knowledge and detection. / Master of Science / Dark patterns are designs on the internet that websites use to trick its users. They may be used to hide advertisements, make the user spend more time or money on their website or more. Our goal was to create a way to help protect anyone on the internet and their information. For this study, we created a program called Dark Pattern Detector that would help the users see different dark patterns that appeared on websites. A study was conducted that had the participants use our program and give us feedback on what they thought of it as well as data on how well it worked. Out of the 40 participants, we found that half the users were unfamiliar with what dark patterns were. Once they completed the study, we saw that the majority of users were able to complete tasks while using our program and gave positive feedback. Seeing the positive feedback and results from our study, we believe that we can help users not get tricked by these patterns and help forward future research on Dark Patterns.

Organizations have recognized usability engineering as a needed step in the development process to ensure the success of any product. As is the case in all competitive settings areas for improvement are scouted and always welcomed. In the case of usability engineering a lot of time, money, equipment, and other resources are spent to gather usability data to identify and resolve usability problems in order to improve their product. The usability data gained from the expenditure of resources is often only applied to the development effort at hand and not reused across projects and across different development groups within the organization. More over, the usability data are often used at a level that forces the organization to only apply the data to that specific development effort. However, if usability data can be abstracted from the specific development effort and analyzed in relation to the process that created and identified the data; the data can then be used and applied over multiple development efforts. The User Action Framework (UAF) is a hierarchical framework of usability concepts that ensures consistency through completeness and precision. The UAF by its nature classifies usability problems at a high level. This high level classification affords usability engineers to not only apply the knowledge gained to the current development effort but to apply the knowledge across multiple development efforts. This author presents a mechanism and a process to allow usability engineers to find insights in their usability data to identify both strengths and weaknesses in their process. In return usability practitioners and companies can increase their return on investment by extending the usefulness of usability data over multiple development efforts. / Master of Science

This manuscript provides an understanding of water conservation related innovations in green buildings, both in terms of what is being selected in design phase and how professionals perceive their experiences with these innovations. The innovations examined include toilets, showers, sinks, plumbing, water heating, appliances, alternative water sources, landscaping, performance monitoring, and user education. It contains a literature review of unanticipated consequences associated with these innovations, and creates a framework for categorizing these based on a synthesis of the literature of unanticipated consequences. A review of certification documents from the Leadership in Energy and Environmental Design (LEED) rating system identifies what landscaping, toilet, and shower innovations are most commonly designed for in LEED certified buildings. These data are also used to identify differences in innovation selection across climate regimes. An internet survey of green building professionals provides a picture of satisfaction with these innovations in practice. It also gives examples of these experiences so that future users can take advantage or take caution as necessary. / Master of Science