Despite intense and wide-spread research in both virtual environments (VEs) and usability, the exciting new technology of VEs has not yet been closely coupled with the important characteristic of usability --- a necessary coupling if VEs are to reach their full potential. Although numerous methods exist for usability evaluation of interactive computer applications, these methods have well-known limitations, especially for evaluating VEs. Thus, there is a great need to develop usability evaluation methods and criteria <i>specifically</i> for VEs. Our goal is to increase awareness of the need for usability engineering of VEs and to lay a scientific foundation for developing high-impact methods for usability engineering of VEs. The first step in our multi-year research plan has been accomplished, yielding a comprehensive multi-dimensional taxonomy of usability characteristics specifically for VEs. This taxonomy was developed by collecting and synthesizing information from literature, conferences, World Wide Web (WWW) searches, investigative research visits to top VE facilities, and interviews of VE researchers and developers. The taxonomy consists of four main areas of usability issues: <i> Users and User Tasks in VEs</i>, <i>The Virtual Model</i>, <i>VE User Interface Input Mechanisms</i>, and <i>VE User Interface Presentation Components</i>. Each of these issues is progressively disclosed and presented at various levels of detail, including specific usability suggestions and context-driven discussion that include a number of references. The taxonomy is a thorough classification, enumeration, and discussion of usability issues in VEs that can be used by VE researchers and developers for usability assessment or simply design. The author can be reached through <a href=http://csgrad.cs.vt.edu/~jgabbard/>http://csgrad.cs.vt.edu/~jgabbard/</a> / Master of Science

The purpose of this study was to explain the safety and usability problems in the residential interior environment of Thai older adults. A sample of 163 retired government employees who live in Bangkok, Thailand were asked to complete a self-administered questionnaire which included questions about housing characteristics, interior environment features, personal information, health condition, and activity level. The mean age of the older adults was 68.1 and ranged from 60 to 93 years. The data were analyzed using frequencies, percentages, and means as descriptive statistics and one-way analyses of variance. The findings revealed that most of the Thai older adults had lived in their own two story detached houses more than ten years and with their family members. The majority of the respondents had vision problems, but almost all could easily perform activities of daily living by themselves and half of them could easily perform instrumental activities of daily living by themselves. Problematic interior environmental features in each area of the home were identified and prioritized. When respondents were divided by age group, significant differences appeared in the degree of difficulty with two safety and usability features in the home. Divided by daily activity levels, respondents revealed significant differences in the degree of difficulty associated with eight safety and usability features. When the homes were broken down to five categories: entrance and stairs, bedroom, bathroom, kitchen, and other interior features of the house, it was these other interior features that seemed to have the most problems in safety and usability. The kitchen had the most problems in safety and usability when compared to other rooms. Based on these findings, design recommendations for Thai housing were developed. / Master of Science

LibX is a browser plugin that allows users to access library resources directly from their browser. Many libraries that wished to adopt LibX needed to customize a version of LibX for their own institution. Most librarians did not possess the necessary knowledge of XML, running scripts and the underlying implementation of LibX required to create customized, functional LibX versions for their own institutions. Therefore, we have developed a web-based tool called the LibX Edition Builder that empowers librarians to create their own customized LibX version (editions), effortlessly. The Edition Builder provides rich interactivity to its users by exploiting the ZK AJAX framework whose components we adapted. The Edition Builder provides automatic detection of relevant library resources based on several heuristics which we have developed, which reduces the time and effort required to configure these resources. We have used sound software engineering techniques such as agile development principles, code generation techniques, and the model-view-controller design paradigm to maximize maintainability of the Edition Builder, which enables us to easily incorporate changing functional requirements in the Edition Builder. The LibX Edition Builder is currently used by over 800 registered users who have created over 400 editions. We have carried out a custom log-based usability evaluation that examined the interactions of our users over a 5 month period. This evaluation has shown that the Edition Builder can dramatically reduce the time needed to customize LibX editions and is being increasingly adopted by the library community. / Master of Science

In recent years, rapid advancements in computer science, including increased capabilities of machine learning models like Large Language Models (LLMs) and the accessibility of large datasets, have facilitated the widespread adoption of AI technology, such as ChatGPT, underscoring the need to design and evaluate these technologies with ethical considerations for their impact on students and teachers. Specifically, the rise of Automated Essay Scoring (AES) platforms have made it possible to provide real-time feedback and grades for student essays. Despite the increasing development and use of AES platforms, limited research has specifically focused on AI explainability and algorithm transparency and their influence on the usability of these platforms. To address this gap, we conducted a qualitative study on an AI-based essay writing and grading platform, with a primary focus to explore the experiences of students and graders. The study aimed to explore the usability aspects related to explainability and transparency and their implications for computer science education. Participants took part in surveys, semi-structured interviews, and a focus group. The findings reveal important considerations for evaluating AES systems, including the clarity of feedback and explanations, impact and actionability of feedback and explanations, user understanding of the system, trust in AI, major issues and user concerns, system strengths, user interface, and areas of improvement. These proposed key considerations can help guide the development of effective essay feedback and grading tools that prioritize explainability and transparency to improve usability in computer science education. / Master of Science / In recent years, rapid advancements in computer science have facilitated the widespread adoption of AI technology across various educational applications, highlighting the need to design and evaluate these technologies with ethical considerations for their impact on students and teachers. Nowadays, there are Automated Essay Scoring (AES) platforms that can instantly provide feedback and grades for student essays. AES platforms are computer programs that use artificial intelligence to automatically assess and score essays written by students. However, not much research has looked into how these platforms work and how understandable they are for users. Specifically, AI explainability refers to the ability of AES platforms to provide clear and coherent explanations of how they arrive at their assessments. Algorithm transparency, on the other hand, refers to the degree to which the inner workings of these AI algorithms are open and understandable to users. To fill this gap, we conducted a qualitative study on an AI-based essay writing and grading platform, aiming to understand the experiences of students and graders. We wanted to explore how clear and transparent the platform's feedback and explanations were. Participants shared their thoughts through surveys, interviews, and a focus group. The study uncovered important factors to consider when evaluating AES systems. These factors include the clarity of the feedback and explanations provided by the platform, the impact and actionality of the feedback, how well users understand the system, their level of trust in AI, the main issues and concerns they have, the strengths of the system, the user interface's effectiveness, and areas that need improvement. By considering these findings, developers can create better essay feedback and grading tools that are easier to understand and use.

Usability inspection methods of user interaction designs have gained importance as an alternative to traditional laboratory-based testing methods because of their cost-effectiveness. However, methods like the heuristic evaluation are ad-hoc, lacking a theoretical foundation. Other, more formal approaches like the cognitive walkthrough are tedious to perform and operate at a high-level, making it difficult to sub-classify problems. This research involves the development and formative evaluation of the Usability Problem Inspection tool — a cost-effective, structured, flexible usability inspection tool that uses the User Action Framework as an underlying knowledge base. This tool offers focused inspections guided by a particular task or a combination of tasks. It is also possible to limit the scope of inspection by applying filters or abstracting lower level details. / Master of Science

Developing economical yet effective methods of incorporating usability engineering as an integral part of software engineering is a primary focus of human-computer interaction (HCI) research. However, much HCI research has focused primarily on inspecting and evaluating applications supporting command-line or graphical user interface (GUI) interaction styles. With the dramatic increase in virtual environment (VE) research in recent years, the HCI community is beginning to place an added emphasis on creating methodologies to ensure usability in VE development. While the demand for VE-specific usability engineering methods and criteria is dire as the amount of money invested by military, government, commercial, and industrial organizations continues to grow, widely accepted methodologies for assessing VE usability are, at this point in time, minimal. There has been a recent increase in research discussing the need of VE-specific usability engineering methodologies, but few research projects have concentrated their efforts on providing such methodologies. Therefore, application developers attempting to apply a user-centered design approach in constructing VEs must often perform largely ad-hoc assessments or in-house evaluations using existing non-VE-specific usability engineering methodologies. The primary focus of this research was to develop a method to guide usability engineering of VEs. The strategy used to develop this usability evaluation method was to modify existing usability evaluation methodologies to support VE development by leveraging the results of previous VE usability research performed at Virginia Tech and elsewhere. The result was a VE-specific usability evaluation methodology that encompasses two existing usability assessment techniques: usability inspections and formative evaluations. We applied this methodology to Crumbs, an immersive visualization VE developed at the National Center for Supercomputing Applications (NCSA). A multi-dimensional framework of VE usability characteristics was a topic of research at Virginia Tech. This framework provided the backbone for VE-specific modifications to the existing usability evaluation techniques proposed in this research. Framework design guidelines permitted usability specialists to perform guidelines-based usability inspections of Crumbs. Results gathered from the guidelines-based usability inspections were used not only to redesign the Crumbs user interface but also drive creation of a formative evaluation plan. Application of the methodology not only uncovered usability issues with Crumbs, but also provided invaluable information regarding the effectiveness of the methodology itself. We conclude this thesis by describing a usability evaluation methodology, called the Modified Concept Book Usability Evaluation Methodology, proposed to improve the usability evaluation methodology performed on Crumbs and other VEs. Our methodology was adapted from an established methodology for performing content analysis on a large volume of qualitative data. Because the realm of VEs is so vast and diverse in application domains and devices, we do not claim that our methodology supports an exhaustive usability evaluation of all VEs. However, the proposed technique is a first attempt at modifying existing usability evaluation methods, and therefore can be used as a launching pad for methodologies to evaluate other aspects of specific VE applications. / Master of Science

Social Network Sites allow individuals to construct a public or semi-public profile within a bounded system, articulate a list of other users with whom they share a connection, and view and traverse their list of connections and those made by others within a system. Such sites are generally centered around a particular activity, such as maintaining social relationships or uploading user created content. Increasingly, niche domains such as education, healthcare and software development have been exploring the creation of social network sites centered around the activities of the domain. This has led to an increasing focus on the processes involved in designing and evaluating these sites. We argue that social network sites require a specialized focus in their design and evaluation on the social utility of the features on the site. We have created two social network sites for niche communities: Colloki, a conversation platform designed for members of local communities; and CATspace, a social repository of Computer Science assignments, designed for use by CS instructors and students. In this thesis, we describe the motivation, design and implementation of these two sites. We provide a formative evaluation of these two sites, wherein we evaluate the usability, and study the perceived social affordances of individual features across the two site. Finally, we discuss future work towards building a framework for evaluating the social utility of Social Network Sites at a formative stage. / Master of Science

Designing global interfaces for users has always been a challenge. This challenge is even greater today with the current trend of globalization, which leads to highly diverse users of the same product. The global audiences for the software and information technology products belong to different countries, different religions, speak different languages, have different life styles, belong to different cultures and have different perceptions and expectations of the same product. A truly global product must inherently accommodate this diversity in order to be effective and successful. A major impediment is that there is very inadequate understanding of the role of culture in user interfaces and how they are built. This lack of understanding is further compounded by the fact that very little empirical work exists regarding the role of culture in usability testing. The objectives of this research are to study and empirically establish the effects of culture on the usability assessment technique of structured interviews. A study was conducted to determine the effects of culture on Indian participants when structured interviews are used in usability testing. The experiment consisted of usability testing of two independent groups of Indian participants by two interviewers; one belonging to the Indian culture and the other to the Anglo-American culture. The findings from the study clearly demonstrate the effects of culture on structured interviews during international user testing. Participants found more usability problems and made more suggestions to the interviewer from their own culture than to the interviewer from a foreign culture. The results of the study prove that culture affects the efficacy of structured interviews during international user testing. / Master of Science

Major research studies have provided support for information scent based usability evaluation and have increased its parent theory's (Information Foraging theory) credibility in the HCI community. These studies have, directly and indirectly, found significant correlations between good information scent and good usability. We would like to investigate its application to less-studied platforms, such as web pages on PDAs and cell phones. The theory itself is not device specific and it implicitly assumes that information scent's importance is universal. However, all studies on the practical application of Information Foraging theory have been conducted with desktop computers. We would like to examine what role information scent plays in interfaces on mobile devices that are limited in usable screen space. For this project, we performed a controlled study with 28 participants on the BBC News web site and its PDA-optimized version. Various usability and information scent related indicators were measured and compared across devices. Contrary to our expectations, we did not find any statistically significant differences between the information scent indicators of the Desktop and PDA sessions and the paths across the devices were highly correlated. / Master of Science

Recently, many full-sized keyboards have been designed to fold in various ways in an attempt to make them more transportable. The flexible membrane keyboard, one type of full sized keyboard, is unique because it is made from silicon rubber, thus it is fully flexible and water resistant. Although a number of flexible keyboard characteristics are the same as standard keyboards (i.e. key size, shape and spacing), key-switch and key clicking mechanisms are inherently different. Since there is little or no existing research on flexible keyboards, there is a current need for data to facilitate design of such keyboards for use. Typing performance and perceived usability of several flexible keyboards that differed in terms of material hardness (hard, medium, or soft) and key contact point shape (circular or square) were studied. The results supported the hypothesis that both typing performance and usability of the flexible membrane keyboard were affected by material hardness and contact point shape. Square shaped contact points led to increased typing speed and decreased error rates, and medium or soft hardness led to increased typing speed. The best flexible keyboard (perceived by participants) in general received neutral usability ratings. However, ratings for mobility and design were much higher than neutral. Overall, subjective and objective measures of performance and usability indicated that flexible keyboards that are made of silicon of a soft or medium hardness and with a square shaped contact points are preferred. / Master of Science