Abstract Many software-intensive products and systems on the market today reveal a poor level of usability despite the availability of knowledge about how to develop good usability, namely user-centred design. Improving the status of user-centred design has proved to be a challenge in product development. An effective step to start out on the road to such improvements is to carry out a current state analysis. Our research problem was to learn how to perform an effective current state analysis, i.e. an assessment, of user-centred design processes to provide a basis for improvement action. As the main part of our research, we carried out five experimental assessments in industrial settings during the years 2000 and 2001. The main result of the research is a novel assessment approach. The objective of the approach is to provide a basis for improvement of the performance of user-centred design in individual product development projects. The particular features of the approach are a new user-centred design process model, a three-dimensional process performance model and implementation of the assessment in the form of a workshop with extensive participation by the project members. The user-centred design process model is method-independent, and consists of six processes that are defined through outcomes. The performance of processes is assessed from the viewpoints of quantity, quality, and integration. Our theoretical examinations show that different assessment categories can be identified depending on the different foci and purposes of an assessment. Moreover, it may even be appropriate to have different assessment approaches for different categories. In the category of our focus and purpose, i.e. 'assessment of user-centred design processes for performance improvement', the focus of an assessment should be in the substance rather than in the management of UCD processes. An assessment approach should be regarded as an artefact and it should be subject to ongoing refinement. An assessment should be an intervention that provides both an effective training session and a meaningful experience to the participants. Clarity in both the concepts and the results seems to take precedence over having standard reference models. In addition, we created a preliminary theory of usability capability. It identifies three dimensions of usability capability: user-centred design infrastructure, performance of user-centred design in product development projects and usability in business strategy. We also propose some new assessment artefacts for other assessment categories and contrast our research experience with established research frameworks.

Doctor of Philosophy / Department of Psychology / John J. Uhlarik / Usability practitioners need effective usability assessment techniques in order to facilitate development of usable consumer products. Many usability evaluation methods have been promoted as the ideal. Few, however, fulfill expectations concerning effectiveness. Additionally, lack of empirical data forces usability practitioners to rely on personal judgments and/or anecdotal statements when deciding upon which usability method best suits their needs. Therefore the present study had two principal objectives: (1) to validate a hybrid usability technique that identifies important and ignores inconsequential usability problems, and (2) to provide empirical performance data for several usability protocols on a variety of contemporary comparative metrics. The User-Centered Hybrid Assessment Tool (User-CHAT) was developed to maximize efficient diagnosis of usability issues from a behaviorally-based perspective while minimizing time and resource limitations typically associated with usability assessment environments. Several characteristics of user-testing, the heuristic evaluation, and the cognitive walkthrough were combined to create the User-CHAT. Prior research has demonstrated that the User-CHAT supports an evaluation within 3-4 hrs, can be used by individuals with limited human factors / usability background, and requires little training to be used competently, even for complex systems. A state-of-the-art suite of avionics displays and a series of benchmark tasks provided the context where the User-CHAT’s performance was measured relative to its parent usability methods. Two techniques generated comparison lists of usability problems – user-testing data and various inclusion criteria for usability problems identified by the User-CHAT, heuristic evaluation, and cognitive walkthrough. Overall the results demonstrated that the User-CHAT attained higher effectiveness scores than the heuristic evaluation and cognitive walkthrough, suggesting that it helped evaluators identify many usability problems that actually impact users, i.e., higher thoroughness, while attenuating time and effort on issues that were not important, i.e., higher validity. Furthermore, the User-CHAT had the greatest proportion of usability problems that were rated as serious, i.e., usability issues that hinder performance and compromise safety. The User-CHAT’s performance suggests that it is an appropriate usability technique to implement into the product development lifecycle. Limitations and future research directions are discussed.

According to Brooke* "Usability does not exist in any absolute sense; it can only be defined with reference to particular contexts." That is, one cannot speak of usability without specifying what that particular usability is characterized by. Driven by the feedback of a reviewer at an international conference, I explore in which way one can precisely specify the kind of usability they are investigating in a given setting. Finally, I come up with a formalism that defines usability as a quintuple comprising the elements level of usability metrics, product, users, goals and context of use. Providing concrete values for these elements then constitutes the investigated type of usability. The use of this formalism is demonstrated in two case studies. * J. Brooke. SUS: A "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, and A. L. McClelland, editors, Usability Evaluation in Industry. Taylor and Francis, 1996.

This Master's Thesis addresses the topic of usability testing of web-based systems. It reviews existing testing methodologies and recommends a collection of test tools and approaches suitable for usability testing. This thesis introduces a new methodology called Tabie for scoring web system under test. The methodology combines both the ease of use of the user interface (syntactic testing) as as well as its content (semantical testing). This approach is well suited for workflows in marketing agencies where content tweaks and its evaluation is done on an ongoing basis in order to attract more viewers and potential customers. The Tabie methodology provides a quantitative and repeatable process, automating the tweak-evaluate iteration.

In the last few years there has been a great development of e-shops and their usage. More and more customers are searching for the opportunity to buy products and services through the internet shops. This is related to the fact that more and more emphasis is placed on their applicability. This thesis deals with the quality of the site and the needs of the application and descibes the use of interactive elements. All the mentioned options are applied in this thesis and compared to the specific e-shop. As a part of the user usability testing there is also a suggestion for corrective actions of discovered deficiencies.

Working together at distance is no easy thing and the usability of the digital workspaces used is of utmost importance. Companies providing these digital workspaces constantly needs to evaluate the usability to find problems and improve their product with design principles for designing interactive systems. The purpose of this study is to contribute with knowledge of the usability of the digital workspace MS Teams. There are several methods for usability evaluation but the most fundamental is to test with real users, a usability test. The method used to evaluate the usability of Microsoft Teams is a usability tests with participants as close to the intended end-users as possible and with post-test interviews after each test. During the usability test the participants were observed whilst using the product and thinking aloud. Thinking aloud is the participant, while using the product, continuously think out loud which gives the researcher an opportunity to understand how the participants view the product and to identify any misconceptions they might have. The main results show that there are several usability problems with Microsoft Teams especially during the log on process and with changing output source. At the same time the results show that sharing files and calling are some of Microsoft Teams strengths in regard to usability and that the perceived usability of Microsoft Teams is high.

The purpose of the study is to investigate college and university students' experience of theusability of Zoom as support for distance education based on usability goals from theory byPreece, Roger and Sharp (2016). To investigate students' experience of the usefulness ofZoom for distance education, semi-structured interviews were conducted with ten studentsfrom different universities and colleges with different educations. The results of theinterviews showed that students' experience of usability at Zoom was good and providedgood support for distance learning based on Preece, Roger and Sharp usability goals (2016).

The video game industry can be ruthless. As a developer, you usually find yourself working in the popular third-party development tools of the time. These tools however might not provide the best usability and quality of life one desires. This can lead to a lot of frustration for the developer, especially when the development enters a crunch period of long and hard work. We believe some of the frustration can be avoided, and we believe this can be done by creating effective, functional and user-friendly integrated development tools specialized for the development environment. In this master's thesis we investigated just that, how integrated game development tools can be designed to be usable in terms of effectiveness and learnability. The investigation was performed by designing and implementing an integrated game development tool. The development of the tool was performed iteratively with user testing between every iteration to find usability defects, allowing the tool to be refined and improved throughout the development process. To finish off the development process, there was a final user test where professional video game developers tried out the tool and then answered a System Usability Scale questionnaire. The System Usability Scale score and task completion rate showed that the final state of the tool can be considered highly usable in terms of effectiveness and averagely usable in terms of learnability. This suggests that involving user testing in the development process is vital for ensuring good usability in the end product.

The digitization in public transport is aimed at all citizens, which means that everyone should be able to use IT-based services in a simple way and obtain available information. The purpose of this study is to identify the usability issues that can occur on “My pages” of county-wide bus service. The purpose is to find out the improvements that can be made to the efficiency and user experience on My pages based on the comments of the test participants.  The research questions in this study are: What usability problems can be identified on My pages? What efficiency improvements can be made on My pages based on test participant feedback and how is the user experience perceived by test participants? The methods used during this study are usability testing using eye-tracking equipment, observation, thinking aloud, and structured interview that includes background questions about the participants.  The results are based on the data collection through the test participants. Eight usability issues were identified, and eight improvement points were revealed. Eye-tracking equipment combined with gaze-plots and observation can be used to see how a usability problem arises. Test participants comments when thinking aloud and interviewing give a better understanding of testing behavior.

With the increased use of online recruitment websites, many organizations have started building up these services to serve various kinds of users. There are some forms of usability evaluations that measure web applications. The inspection method is the most widely used, especially the heuristics evaluation. However, when facing substantial market demands, there is no set of heuristics to guide web designers on how to offer users a good user experience.  Therefore, developing usability heuristics for online recruiting websites is very necessary. According to existing research, Nielsen ten heuristics are the most widely used and well known. However, with the continual emergence of new technologies, these ten heuristics cannot continue to satisfy designers who are looking for more focused heuristics that apply to a specific domain. There exists no working strategy for developing specialized guidelines to ensure specialized heuristics actually work better than using general heuristics.  This thesis aims to develop usability heuristics for online recruitment website domains. Even though there are no appropriate theories or models for developing heuristics in a specific domain, we established a proposal of 21 usability heuristics for online recruitment based on the existing heuristics and methodologies to develop new specialized heuristics. These new heuristics offer evaluators smoother operations with higher efficiency (problems/time spent) in heuristics evaluation.