This work explores the utilization of scenarios in an iterative usability engineering process for the development of a next-generation multilocus sequence typing (MLST) tool. The following three research question were investigated during the usability process: (1) what are the differences in the elicited requirements as scenarios move further from extant work practices, (2) what are the differences in the elicited requirements between structured and free-form scenario groups, and (3) are participant-developed scenarios from the scenario-based interviews effective for use as tasks in formative usability evaluation. Scenario-based interviews were conducted to collect relevant work-practice information and domain knowledge from two user classes. Requirements distilled from the scenarios and complementary interview questions informed the design of multiple iterations of the tool. A formative usability evaluation was conducted on the second iteration of the tool with the same participants. Resulting requirements from the scenario-based interviews suggest that proposing scenarios beyond current work practices overwhelmed and confused participants, and therefore worked against requirements generation. Conversely, a less structured scenario-based interview scheme yielded a greater quantity of requirements, and specifically produced more creative requirements. Participant-developed scenarios from the scenario-based interviews were ultimately useful as benchmark tasks in the formative usability evaluation because they were intricate enough to afford meaningful interaction with the interface, while still being completable by both user classes. This research helps to provide a greater understanding of the utilization of novel scenario styles and methodologies, thereby providing support for the continued investigation into scenario use for a variety of applications. / Master of Science

The usability of an application often plays an important role in determining its success. Accordingly, organizations that develop software have realized the need to integrate usability engineering into their development lifecycles. Although usability practitioners have successfully applied usability engineering processes to increase the usability of user-interaction designs, the literature suggests that usability practitioners experience a number of difficulties that negatively impact their effectiveness. These difficulties include identifying and recording critical usability data, understanding and relating usability data, and communicating usability information. These difficulties are particularly pronounced for novice usability practitioners. With this dissertation, I explored approaches to address these difficulties through tool support for novice usability practitioners. Through an analysis of features provided by existing tools with respect to documented difficulties, I determined a set of desirable tool features including usability problem instance records, usability problem diagnosis, and a structured process for combining and associating usability problem data. I developed a usability engineering tool, the Data Collection, Analysis, and Reporting Tool (DCART), which contains these desirable tool features, and used it as a platform for studies of how these desirable features address the documented difficulties. The results of the studies suggest that appropriate tool support can improve the effectiveness with which novice usability practitioners perform usability evaluations. More specifically, tool support for usability problem instance records helped novice usability practitioners more reliably identify and better describe instances of usability problems experienced by participants. Additionally, tool support for a structured process for combining and associating usability data helped novice usability practitioners create usability evaluation reports that were of higher quality as rated by usability practitioners and developers. The results highlight key contributions of this dissertation, showing how tools can support usability practitioners. They demonstrate the value of a structured process for transforming raw usability data into usability information based on usability problem instances. Additionally, they show that appropriate tool support is a mechanism for further integrating usability engineering into the overall software development lifecycle; tool support addresses the documented need for more usability practitioners by helping novices perform more like experts. / Ph. D.

This research examines the healthcare services ecosystem and the impact and role service interventions made by providers and patients have on this ecosystem. Each area has an important role in contributing to the value and sustainability of the ecosystem. Healthcare, as a community service, requires a minimum of two counterparts: the providers and the customers, in this case the patients. Healthcare is a unique ecosystem because often the customers are not conscious of the interplay of the ecosystem but are reliant upon the system for their health and wellbeing. The first section of this dissertation examines the effects that occur in the healthcare ecosystem when part of the system experiences a disaster and the impact and role of other areas of the system in response to the disaster, particularly regarding the resilience. Similar to a biological ecosystem that is undergoing a flood, in the healthcare services ecosystem if too many patients present to the Emergency Department (ED) at the same time disaster level overcrowding will occur. We aim to measure the resilience of the healthcare ecosystem to this disaster level overcrowding. The second section of this dissertation examines how the components of the healthcare ecosystem maintain sustainability and usability. Healthcare professionals are assessed regarding their ability to maintain the healthcare ecosystem, with a specific focus on what occurs after patients are in the hospital system. To examine the ability of the healthcare professionals to maintain the ecosystem we analyze the usability and adaptability of the electronic health record and the professional's workflows to determine how they use this tool to sustain the healthcare ecosystem. The third section of this dissertation examines patient self-management and the influence this has on the healthcare ecosystem. Much of the management of health in patients, particularly those with chronic illnesses, occurs outside of the hospital, thus examining this aspect of self-care provides insight on the overall system. This research examines patients with a chronic illness and their use of online health communities, with a particular focus on their reciprocal behaviors and the impact this support system has on their overall health state. By examining these aspects of the healthcare services ecosystem, we can better improve our understanding of these phenomena. / Ph. D.

In the electric utility industry, displays provide power system operators with information on and the status of the system, who then make decisions on how to maintain the safety, the reliability and the efficient operation of the utility generation and transmission grid based on that information. Complexity of the data presented and the display itself can lead to errors or misjudgments that can cause power system operators to make unwise decisions. The primary goal of this research was to develop a method to quantify display complexity for select displays used by system operators when operating the electric generation and transmission grids. Three studies were performed: (1) complexity measure development, (2) validation of the measure using usability and situation awareness (SA) techniques, and (3) display revisions based on complexity measure findings. Fifteen 15 different complexity metrics were originally considered (additive models, multiplicative models, and combination models with five different weighting schemes). The additive model with equal weighting was found to be the most sensitive in differentiating displays and was used in the later studies. For the validation study, system operators were asked to complete a usability questionnaire and a paper-based SA test using the current displays. Correlation and scatter plot analyses was used to determine if the complexity metric and usability and SA scores were related. Results of the validation study indicated that usability and SA scores for the studied displays were not well correlated with the complexity metric. In study 3, the highest and lowest scoring displays were redesigned with an emphasis on maintaining functionality but reducing aspects of complexity that were driving the complexity score. Systems operators again completed the usability and SA testing using the redesigned displays and again correlation analysis was performed. As was the case with study 2, usability scores were not correlated with the complexity metric; however, SA scores were significantly correlated. The complexity metric developed here can be used to quantify the complexity in a display and identify redesign opportunities to reduce non-essential information, as displays that are less complex should result in improved operator performance and satisfaction with the display.

Die vorliegende Bachelorarbeit befasst sich mit der Untersuchung der Usability der Betaversion des neuen Onlinekatalogs der Deutschen Nationalbibliothek anhand von Usability-Tests. Das Ziel der Arbeit besteht darin, die Benutzungsfreundlichkeit des Katalogs zu evaluieren und mögliche Verbesserungspotenziale aufzuzeigen, um den Benutzenden eine optimale Suche durch intuitive Recherchemöglichkeiten wie Filter zu ermöglichen. Die Untersuchung beschränkt sich auf die Recherche in der Betaversion. Der theoretische Teil der Arbeit umfasst eine detaillierte Beschreibung der grundlegenden Konzepte und Methoden im Bereich der Usability-Untersuchungen. Im praktischen Teil werden Usability-Tests durchgeführt, bei denen Probanden verschiedene Aufgaben im Onlinekatalog ausführen. Die Ergebnisse des Tests werden in Bezug auf Usability-Probleme und Verbesserungspotenziale hinsichtlich der Recherchewege ausgewertet. Es werden konkrete Lösungsvorschläge entwickelt, um die Benutzungserfahrung im Onlinekatalog der Deutschen Nationalbibliothek zu optimieren. Die vorliegende Arbeit soll dazu beitragen, die Usability des neuen Onlinekatalogs zu verbessern und den Benutzenden ein effizientes und angenehmes Sucherlebnis zu bieten.:1 Einleitung 1.1 Ziel und Aufgabe der Bachelorarbeit 1.2 Überblick über den Aufbau der Arbeit 2 Usability 2.1 Definition von Usability 2.2 Bedeutung der Usability für OPACs von Bibliotheken 2.3 Deutsche Nationalbibliothek 2.4 Methoden der Usability-Evaluation 2.4.1 DIN EN ISO 9241-110 2.4.2 Expertenbewertung 2.4.3 Eye-Tracking 2.4.4 A/B-Tests 2.4.5 Interviews 2.4.6 Usability-Tests 2.4.7 Fazit 3 Umsetzung der Evaluationsmethode zur Bewertung der Usability 3.1 Auswahl der Teilnehmenden 3.2 Fragebogen 3.3 Pretest 3.4 Durchführung der Evaluation 4 Ergebnisse der Usability-Evaluation 5 Fazit und Empfehlungen 6 Literaturverzeichnis 7 Anhang 7.1 Untersuchungsdesign 7.2 Fragebogen 7.3 Fragebögen der Teilnehmenden 7.4 Gesprächsnotiz des Gruppengesprächs mit den Teilnehmenden 7.5 Tabelle Auswertung 8 Selbstständigkeitserklärung

The Internet is gradually expanding to many new devices, in addition to its original native environment that was the Personal Computer. This wave started with mobile devices and as we enter the Internet of Things era, connectivity is possible from cars to light switches. One of the first devices, to follow mobiles, is the Television. Connectivity and two-way interaction on the TV device has in fact started even in the first days of the medium, but had failed to make it widely available, due to technological limitations of the past. Now, this has changed, with the Smart TV devices that can utilize the fast internet connections that are available in most developed countries. However, even though the technology and devices are now widely available there are still challenges in order to make the vast Internet and web content available in the Smart TVs. These challenges, have a familiar resemblance to what happened a few years ago, when internet connectivity was introduced on the mobile phone. Although, it was "feasible" to access any webpage from your mobile device, the experience for the user was often very frustrating, due to many factors, that derived from the fact that the web was designed for large screens and mice of the desktop computers, not the small touch screens and limited hardware of the phone. Nonetheless, these obstacles were successfully overcome, by introducing techniques and methodologies (e.g. Responsive Web Design) to make the web more mobile-friendly and also work from the manufacturers to improve their devices to this direction as well. The success of these actions is now evident, since the access to the web from mobile devices has surpassed the PC, and it is now a standard practice for every new website to be mobile-friendly. In this research work, we will attempt to do one very significant step towards this direction for the Smart TV. In other words, to discover what has to be done to make the web more TV-friendly. To do this, we explore many different TV devices from several manufacturers and see their similarities and differences. We explore numerous user studies and surveys to discover what is the problem in the Web experience on the TV, so that we can propose solutions to make web content TV-friendly. Based on these findings, we design prototypes and put them to the test on different devices and userevaluation. Finally, we propose a set of guidelines, that web designers can apply on their websites to make them TV-friendly, in the hope to introduce the first step towards a friendlier internet era for the TV.

Usability is one of the important quality attributes. Open source software products are well known for their efficiency and effectiveness. Lack of usability in OSS (Open Source Software) products will result in poor usage of the product. In OSS development there is no usability team, and one could therefore expect that the usability would be low for these products. In order to find out if this was really the case we made a usability evaluation using a questionnaire for four OSS products. The questionnaire was based on a review of existing literature. This questionnaire was presented to 17 people who are working with open source products. This evaluation showed that the overall usability was above average for all the four products. It seems, however, that the lack of a usability team has made the OSS products less easy to use for inexperienced users. Based on the responses to questionnaire and a literature review, a set of guidelines and hints for increasing the usability of OSS products was defined.

In recent years, interactive systems developers have increasingly included usability engineering and interaction design as an integral part of software development. With recognition of the importance of usability come attempts to structure this new aspect of system design, leading to a variety of processes and methodologies. Unfortunately, these processes have often lacked flexibility, completeness and breadth of coverage, customizability, and tool support. This thesis shows the development of a process model, that we call LUCID/Star*, which addresses and overcomes the characteristics lacking in existing methodologies and an evaluation of its application in a real-world development environment. To demonstrate the goal of this thesis, we have used a combination of empirical and analytical evidence. The (LUCID/Star)* process model for usability engineering grew out of the examination, adaptation, and extension of several existing usability and software methodologies. The methods that most greatly impacted the creation of (LUCID/Star)*were the LUCID Framework of interaction design, the Star Life Cycle of usability engineering, and the Waterfall and Spiral models of Software engineering. Unlike most of these, we have found that a sequence of cycles (each of which produces a product evolution) is a more effective analogy for the interaction development process. A sequence of cycles is more modular and makes it easier to focus on each cycle separately. Working with Optim Systems, Inc. in Falls Church, VA we instantiated the process model and introduced it as a process to develop a web-based device management system. (LUCID/Star)* performed remarkably in the Optim case, overcoming the tight constraints of budget and schedule cuts to produce an excellent prototype of the system. / Master of Science

The need for cost-effective usability evaluation has led to the development of methodologies to support the usability practitioner in finding usability problems during formative evaluation. Even though various methods exist for performing usability evaluation, practitioners seldom have the information needed to decide which method is appropriate for their specific purpose. In addition, most methods do not have an integrated relationship with a theoretical foundation for applying the method in a reliable and efficient manner. Practitioners often have to apply their own judgment and techniques, leading to inconsistencies in how the method is applied in the field. Usability practitioners need validated information to determine if a given usability evaluation method is effective and why it should be used instead of some other method. Such a desire motivates the need for formal, empirical comparison studies to evaluate and compare usability evaluation methods. In reality, the current data for comparing usability evaluation methods suffers from a lack of consistent measures, standards, and criteria for identifying effective methods. The work described here addresses three important research activities. First, the User Action Framework was developed to help organize usability concepts and issues into a knowledge base that supports usability methods and tools. From the User Action Framework, a mapping was made to the Usability Problem Inspector; a tool to help practitioners conduct a highly focused inspection of an interface design. Second, the reliability of the User Action Framework was evaluated to determine if usability practitioners could use the framework in a consistent manner when classifying a set of usability problems. Third, a comprehensive comparison study was conducted to determine if the Usability Problem Inspector, based on the User Action Framework, could produce results just as effective as two other inspection methods (i.e., the heuristic evaluation and the cognitive walkthrough). The comparison study used a new comparison approach with standards, measures, and criteria to prove the effectiveness of methods. Results from the User Action Framework reliability study showed higher agreement scores at all classification levels than was found in previous work with a similar classification tool. In addition, agreement using the User Action Framework was stronger than the results obtained from the same experts using the heuristic evaluation. From the inspection method comparison study, results showed the Usability Problem Inspector to be more effective than the heuristic evaluation and consistent with effectiveness scores from the cognitive walkthrough. / Ph. D.

Remote usability evaluation is a usability evaluation method (UEM) where the experimenter, performing observation and analysis, is separated in space and/or time from the user. There are several approaches by which to implement remote evaluation, limited only by the availability of supporting technology. One such implementation method is RECITE (the REmote Critical Incident TEchnique), an adaptation of the user-reported critical incident technique developed by Castillo (1997). This technique requires that trained users, working in their normal work environment, identify and report critical incidents. Critical incidents are interactions with a system feature that prove to be particularly easy or difficult, leading to extremely good or extremely poor performance. Critical incident reports are submitted to the experimenter using an on-line reporting tool, who is responsible for their compilation into a list of usability problems. Support for this approach to remote evaluation has been reported (Hartson, H.R., Castillo, J.C., Kelso, J., and Neale, W.C., 1996; Castillo, 1997). The purpose of this study was to quantitatively assess the effectiveness of RECITE with respect to traditional, laboratory-based applications of the critical incident technique. A 3x2x 5 mixed-factor experimental design was used to compare the frequency and severity ratings of critical incidents reported by remote versus laboratory-based users. Frequency was measured according to the number of critical incident reports submitted and severity was rated along four dimensions: task frequency, impact on task performance, impact on satisfaction, and error severity. This study also compared critical incident data reported by trained users versus by usability experts observing end-users. Finally, changes in critical incident data reported over time were evaluated. In total, 365 critical incident reports were submitted, containing 117 unique usability problems and 50 usability success descriptions. Critical incidents were classified using the Usability Problem Inspector (UPI). A higher number of web-based critical incidents occurred during Planning than expected. The distribution of voice-based critical incidents differed among participant groups: users reported a greater than expected number of Planning incidents while experts reported fewer than expected Assessment incidents. Usability expert performance was not correlated, requiring that separate analyses be conducted for each expert data set. Support for the effectiveness in applying critical incidents to remote usability was demonstrated, with all research hypotheses at least partially supported. Usability experts gave significantly different ratings of impact on task performance than did user reporters. Remote user performance versus laboratory-based users failed to reveal differences in all but one measure: laboratory-based users reported more positive critical incidents for the voice interface than did remote users. In general, the number of negative critical incidents decreased over time; a similar result did not apply to the number of positive critical incidents. It was concluded that RECITE is an effective means of capturing problem-oriented data over time. Recommendations for its use as a formative evaluation method applied during the latter stages of product development (i.e. when a high fidelity prototype is available) are made. Opportunities for future research are identified. / Master of Science