This case study investigated how to evaluate users’ experiences with physical products with a small sample size in a reliable way and how to provide actionable insights for future decisions about design and practice. Through an improving case study performed in collaboration with ASSA ABLOY, a global leader in access solutions, this study specifically focused on interactions with locks. Three major activities were performed and used to investigate how to conduct usability tests with a small sample size and to explore possible measures that can help increase reliability. Altogether 13 participants participated in this case study through two sets of tests and one final workshop. These activities had the purpose to investigate, to suggest and to evaluate how to best capture users’ experiences with locks using Single-Ease Question, Nasa Task Load Index instrument and Product Reaction Cards. The results showed that there are many different measures to apply to increase reliability in test design whilst using a small sample size, for example to mix methods and to counterbalance, both metrics and tasks, to both deepen the understanding of the experiences and to decrease the risks of possible biases. Further results also showed that qualitative and quantitative methods provide different insights about user’s experiences with respect to detailed and general knowledge respectively. It also showed that a combination of the two provides deeper insights than what one method provides alone and that they help validate individual findings.

Denna rapport avser att undersöka för- och nackdelar med att utveckla ett för ändamålet specialanpassat system eller utnyttja de standardsystem, som i det aktuella fallet tryckerikoncernen JMS redan äger och använder. Genom en mer noggrann definiering av ickefunktionella krav på bland annat användbarhet har en specialanpassad prototyp framställts. I projektet användes metoder för kravinsamling som ledde till en kravspecifikation och i sin tur slutade med en färdig prototyp. Prototypen jämfördes med två standardsystem genom användartester och intervjuer. Prototypen visade sig leva upp till de krav som upptäcktes. Majoriteten av testpersonerna föredrog ett enklare avskalat system, vilket prototypen upplevdes som, före system med mycket information och funktioner. Med tanke på kostnaden rekommenderas dock JMS att försöka strukturera upp ett av de i företaget befintliga systemen och använda detta istället för att utveckla och implementera ytterligare ett nytt system i företaget. / This report intends to explore the pros and cons of developing a customized system or use the standard system, which in this case printing JMS Group already owns and uses. With a more accurate definition of non-functional requirements such as usability, has a custom-made prototype been built. The project used methods of requirements gathering that led to a specification and ended with a finished prototype. The prototype was compared with two standard systems through user testing and interviews. The prototype proved to live up to the requirements discovered. The majority of test subjects preferred a simpler clean system, which the prototype was, to systems with a lot of information and functions. Given the cost JMS was recommended to structure one of the systems that they already owns and uses instead of developing and implementing a new system further in the company.

Previous usability evaluation method (UEM) comparison studies have noted an evaluator effect on problem detection in heuristic evaluation, with evaluators differing in problems found and problem severity judgments. There have been few studies of the evaluator effect in usability testing (UT), task-based testing with end-users. UEM comparison studies focus on counting usability problems detected, but we also need to assess the content of usability problem descriptions (UPDs) to more fully measure evaluation effectiveness. The goals of this research were to develop UPD guidelines, explore the evaluator effect in UT, and evaluate the usefulness of the guidelines for grading UPD content. Ten guidelines for writing UPDs were developed by consulting usability practitioners through two questionnaires and a card sort. These guidelines are (briefly): be clear and avoid jargon, describe problem severity, provide backing data, describe problem causes, describe user actions, provide a solution, consider politics and diplomacy, be professional and scientific, describe your methodology, and help the reader sympathize with the user. A fourth study compared usability reports collected from 44 evaluators, both practitioners and graduate students, watching the same 10-minute UT session recording. Three judges measured problem detection for each evaluator and graded the reports for following 6 of the UPD guidelines. There was support for existence of an evaluator effect, even when watching pre-recorded sessions, with low to moderate individual thoroughness of problem detection across all/severe problems (22%/34%), reliability of problem detection (37%/50%) and reliability of severity judgments (57% for severe ratings). Practitioners received higher grades averaged across the 6 guidelines than students did, suggesting that the guidelines may be useful for grading reports. The grades for the guidelines were not correlated with thoroughness, suggesting that the guideline grades complement measures of problem detection. A simulation of evaluators working in groups found a 34% increase in severe problems found by adding a second evaluator. The simulation also found that thoroughness of individual evaluators would have been overestimated if the study had included a small number of evaluators. The final recommendations are to use multiple evaluators in UT, and to assess both problem detection and description when measuring evaluation effectiveness. / Ph. D.

Traditional in lab usability evaluation has been used as the 'standard' evaluation method for evaluating and improving usability of software user interfaces (Andre, Williges, & Hartson, 2000). However, traditional in lab evaluation has its drawbacks such as availability of representative end users, high cost of testing and lack of true representation of a user's actual work environment. To counteract these issues various alternative and less expensive usability evaluation methods (UEMs) have been developed over the past decade. One such UEM is the Remote Usability Evaluation method. Remote evaluation is a relatively new area and lacks empirical data to support the approach. The need for empirical support was addressed in this study. The overall purpose of this study was to determine the differences in the effectiveness of the two evaluation types, the remote evaluation approach (SREM) and the traditional evaluation approach, in collecting usability data. This study also compared the effectiveness between the two methods based on user type, usability novice users and usability experienced users. Finally, the hypothesis that users, in general, will prefer the remote evaluation approach of reporting to the traditional in-lab evaluation approach was also tested. Results indicated that, in general, the synchronous remote approach is at least as effective as the traditional in lab usability evaluation approach in collecting usability data across all user types. However, when user type was taken into consideration, it was found that there was a significant difference in the high severity negative critical incident data collected between the two approaches for the novice user group. The traditional approach collected significantly more high severity negative critical incident data than the remote approach. Additionally, results indicate that users tend to be more willing to participate in the same approach as the one they participated previously. Recommendations for usability evaluators for conducting the SREM approach and areas for future research are identified in the study. / Master of Science

The Agile software development model is driven by "learning by doing" and rejects Big Design Up Front (BDUF) for that reason. User-Centered Design (UCD) on the other hand requires a more holistic view to be able to create a usable user interface and in the end create a good user experience. Finding a balance between the incremental development and the need for a more comprehensive view of the user interface is therefore the key to usability in Agile software development. The objective of this master thesis was to construct a framework on how to combine UCD and Agile development in general, and specifically for the web based tool, called Alo, at the IS/IT department of Com Hem AB, Sweden. The results of this thesis was that the process of integrating User-Centered Design in Agile software development first of all needs a familiar starting point for both usability experts and developers. This can be achieved with what Desirée Sy describes as “Cycle Zero”, to let usability experts perform initial research ahead of implementation. Designing one sprint ahead should later converge to a more synchronized process where requirements and sketches of the interface are put together, with the help of developers, just in time for the implementation. This does not only prevents waste in the form of documentation and miscommunication associated with hand-offs, but also makes the implementation more purposeful and fun for developers.Secondly, build prototypes early in the process to create a holistic vision of the finished product and to test concepts in usability tests early. Thirdly, create shared understanding (within the development team as well as with outside stakeholders) of user needs by involving the entire team in usability testing. Critical to the success of all the above is that all outside stakeholders understands the Agile process and respects that the team is a self-organizing unit that solves problems within a set of given boundaries, rather than a code factory that feeds on specification documents.

Achieving the feeling of a personal connection over the internet is difficult but at the same time more relevant than ever with the world moving more and more towards digital meetings and online events. This study investigates how an exhibition area at an online conference could be created to provide a good user experience as well as provide value for the exhibitors at the conference. The study was performed through interviews, three stages of prototyping as well as usability testing of these prototypes to gather data and feedback on what could be done to achieve this goal. Through the principles of social presence, human centered, and user centered design, a prototype of how such a solution could look was developed. As a result, the test users felt that they would feel welcome in the digital booth and that they felt a greater incentive to join conversations with the exhibitors. On the other end, the exhibitors also felt like this solution would provide value in the form of monitorable gathered data that could be followed up on to investigate the value that the conference has given them. This concludes that engaging and valuable online exhibitions can be created as long as careful thought is put into designing them with the user and human in mind.

<p>Chronic low back pain is the second most prevalent chronic condition in Canadian primary care settings. The treatment and diagnosis of chronic pain is challenging for primary care clinicians. Their main challenges are lack of knowledge and their approach toward assessing and treating pain. Evidence based guidelines have been developed for neuropathic pain and low back pain.</p> <p>CDSSs for chronic diseases are becoming popular in primary care settings as a mean to implement CPGs. A CDSS prototype for diagnosis and treatment of chronic, non-cancer pain in primary care was developed at McMaster University. It is evident that poor usability can hinder the uptake of health information technologies.</p> <p>The objective of this study was to test the usability of Pain Assistant using think aloud protocols with SUS scores in 2 iterations. In this study 13 primary care providers including family physicians, nurse practitioners and residents used Pain Assistant to complete 3 different patient case scenarios. Participants were asked to comment on both barriers and facilitators of usability of Pain Assistant. Additionally time to complete patient case scenarios was calculated for each participant. A comparison questionnaire gathered user preference between introducing CPGs in paper format and computerized decision support system.</p> <p>This study showed that iterative usability testing of the Pain Assistant with participation of real-end users has the potential to uncover usability issues of the Pain Assistant. Problems of user interface were the main usability barrier in first testing iteration following by problems of content. Changes were made to system design for second round based on the issues came up in the first iteration. However, because of time constrains not all the changes were implemented for second round of testing. Most of the refinements were to resolve user interface issues. In the second iteration, the problems with the content of Pain Assistant were the major barrier. The changes to the system design were successful in resolving user interface problems since the changed issues did not come up again in second round. Pain Assistant had an above the average usability score however no significant changes seen in SUS score. The time needed to complete tasks remained identical in both iterations. In addition, participants preferred to have CPGs in electronic formats than paper. Further study after implementing all the system changes needed to determine the effectiveness of system refinements.</p> / Master of Science (MSc)

Vårt samhälle har genomgått stora förändringar då det rör sig teknik. Många områden i dagens samhälle blev digitaliserade och Vården, vår hälsa blev inte ett undantag. Landstinget tillhandahåller ett IT- system eller e- tjänst, som möjliggör patienten att boka/ omboka tid, förnya recept, beställa sina patientjournaler och med mera. E- tjänsten Mina Vårdkontakter (MVK) skapades för drygt 10 år sedan. Tjänsten vänder sig till alla medborgare, oavsett dator- och Internetanvändningserfarenhet, kön och ålder. Eftersom alla medborgare inte är lika vana IT- användare och därför ett sådant system bör vara så användarvänlig som möjligt. Detta arbete syftar på att hitta brister i MVK och hitta förslag på förbättringar utifrån användbarhetsprinciper. Metoder som användes i detta arbete är kombination av kvalitativ och kvantitativ metod. Därför utvärderade jag e- tjänsten MVK, genom att genomföra ett antal användbarhetstester. Testpersoner i detta arbete var både män och kvinnor i olika åldrar, har olika erfarenhet av dator- och Internetanvändning. Mina testfrågade fick ett antal enkla uppgifter, som jag ansåg kan ha några brister. Det svåraste av alla uppgifter var att Hitta MVK på lg.se webbsida. Men det var inte heller lätt att hitta hjälp på MVK. Den bästa tjänsten av alla som finns på MVK var att avboka/ omboka tiden och förnya recept- funktionen. Med hjälp av Time- on- task observerade jag också att oerfarna personer tar alltid längre tid att utföra vissa uppgifter jämfört med experten och med erfarna testpersoner. Och det är just för dem man ska förbättra e- tjänsten för att det skulle vara så enkelt som möjligt att använda e- tjänsten och personer skulle trivas med systemet. Efter genomfört arbete identifierades förslag på förbättringar av e- tjänsten, samt man kunde ge förslag på framtida forskning. Framtid forskning kan vara en ny utvärdering av bara en grupp oerfarna användare i olika åldrar. Man kan också undersöka hur den grafiska designen ser ut och hur det går att navigera i MVK. Eller så kan man göra en ny utvärdering efter den planerade uppdateringen av MVK som skulle ske om drygt en månad, enligt MVK- utvecklare. / Our society has undergone significant changes as new techniques came into our lives. Many areas of today’s society have digitalized in very short period and even healthcare sector is not exception. The County Council (Landstinget) provides an IT- system or e- service (MVK), which allows patients to book/ cancel appointments, renew prescriptions, order Patient Medical Records and much more. E- service was created just over 10 years ago. The service is aimed to all citizens, regardless of Computer- and Internet experience, gender and age. Since not all citizens are experienced in usage of IT, such system should be as easier and as user friendly as possible. This work is aimed to find weaknesses in MVK and find suggestions for improvement based on usability principles. Methods used in this work the combination of qualitative and quantitative methods. Therefore, I evaluated the service MVK, by implementing a number of usability tests. Test persons in this work were both men and women of different ages and different Computer- and Internet experiences. Test persons got a number of simple tasks, in which I felt we might find some faults. The most difficult of all was to find MVK on lg.se website. But it was not easy either to find Help on MVK. The best service of all available on MVK is to cancel/ reschedule the appointment and renew- prescription function. By using Time- on- task, I observed that inexperienced computer and internet users take longer time to perform certain tasks, compared with the expert and with other test takers. And that is right for them we should improve the system so that it would be easy for them to use it. In this work I could identify suggestions to improve the e- service, and could give suggestions to future research. Future research may be a reassessment of just a group of inexperienced users of different ages. We can also examine how the graphic design looks like and how it is to navigate on MVK. Or we can also re- evaluate e- service after the planned updates of MVK that would be carried out in a month, according to MVK developers.

Healthcare is an important part of life in most societies that attract a significant amount of public investment. Primary healthcare is a fundamental branch of the healthcare system where patients and doctors initially meet. Family Medicine Information Systems are developed in an effort to ease the daily work of family doctors with the help of information technology. Such systems are generally used for handling critical tasks such as managing health records of patients, monitoring pregnancy and keeping track of children&rsquo / s vaccination. Like any medical information technology, the usability of such systems is a vital concern for enabling efficient and effective primary healthcare operations. Family Medicine is a recently established practice in Turkey and there are a number of systems in service to aid the daily work of family doctors. However, none of these systems have been subjected to a systematic usability analysis. In this study, a usability analysis of a popular Family Medicine Information System used in Turkey is conducted. By combining several usability evaluation techniques, the study identified several important usability issues and provided recommendations for further improving the system. The main usability issue observed in the system was the overall complexity of the information presented at the main interface that often confused and misled the users. In order to address this problem, it is suggested that features related to the most frequent family medicine operations should be placed on the main screen, whereas remaining features should be organized under auxiliary pages with clear navigation aids.

This thesis examines the user-centered design methods and methodology inherent to designing and testing a web-based Dreamweaver 8 tutorial for undergraduate and graduate students who enroll in certain English rhetoric and composition courses at Georgia State University. The tutorial’s three interfaces were rhetorically designed to support three corresponding types of user—novices, intermediates, and experts— whose familiarity with Dreamweaver and student web space determined their starting point of interaction with the artifact. Three usability tests examined each interface based on four usability attributes. Findings revealed the novice and expert interfaces to be usable, while the intermediate interface was more problematic. The analysis of findings indicated the advanced documentation theory to be sound; however, the practical implementation of the theory to this artifact was comparatively ineffective. More research is suggested for determining whether a multimodal tutorial design is the most useful and usable for the target audience(s).