Return to search

Utilizing Machine Learning Methods for Usability Evaluation in Learning Management Systems

The concept of usability refers to a user's capability to interact with a system to fulfill goals in terms of task completion (effectiveness), time measurement (efficiency), and positive attitude (satisfaction). The strategy for usability evaluation in software systems usually involves questionnaires, user testing, and heuristics. Although these methods have been widely used due to several benefits, there are challenges related to time consumption and embedded bias. In response to these challenges, this work proposes a hybrid approach based on usability questionnaire answers and machine learning algorithms to predict usability scores. We describe three different experiments with features extracted from a Learning Management System. These features were applied in the Machine Learning algorithms Linear Regression, Decision Trees, Random Forest, and Neural Networks in three experiments. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements for better performance of the System Usability Scale and UseLearn scores prediction. This approach has potential as a reliable predictive tool for usability scores, which would help create software systems that better satisfy users' needs. / Master of Science / Instructors and students have used online platforms known as Learning Management Systems (LMSs) to improve learning and satisfaction. Students need to achieve their learning goals by interacting with these systems. To achieve these goals, usability evaluation involves ensuring that LMSs attain effectiveness (task completion), efficiency (time measurement), and satisfaction (positive attitude). Usability evaluation usually follows questionnaires, user testing of the LMS, and expert reviews. Although these methods are widely used due to several benefits, they face challenges related to trying these software systems multiple times until the system satisfies student needs and human subjectivity perception. To face these challenges, promote student engagement with the system, and create a better design in the LMS courses, we propose a hybrid approach based on data, questionnaire answers, and machine learning algorithms to predict usability scores. We evaluated this approach through a case study with data collected from undergraduate students at Virginia Tech. The results showed different advantages and drawbacks of machine learning performance. The approach contributes to the engineering and computing education field by providing a reliable predictive tool for usability scores to improve the student learning experience and the features of the LMS.

Identiferoai:union.ndltd.org:VTETD/oai:vtechworks.lib.vt.edu:10919/118974
Date14 May 2024
CreatorsTorres Molina, Richard Andres
ContributorsComputer Science and#38; Applications, Seyam, Mohammed Saad Mohamed Elmahdy, Fox, Edward A., McCrickard, Donald Scott
PublisherVirginia Tech
Source SetsVirginia Tech Theses and Dissertation
LanguageEnglish
Detected LanguageEnglish
TypeThesis
FormatETD, application/pdf
RightsIn Copyright, http://rightsstatements.org/vocab/InC/1.0/

Page generated in 0.002 seconds