This study addresses faculty perceptions of how the features of existing learning management systems (LMS) currently and potentially enhance their assessment of student work. Within this type of technology, a selection of a couple of main systems, namely Blackboard and Moodle, was examined. A critical analysis of the exiting literature on the adoption of assessment features within the classrooms for formative purposes is presented. A mixed methods research design was used in order to evaluate the effectiveness of formative assessment tools in LMS. A survey was electronically distributed to professors in two mid-sized universities in Eastern Ontario, Canada to gather data in the first phase. The second phase of data collection entailed interviewing a subset of the professors after analysing the first set of data. Additional analysis was conducted in order to identify the factors that can elevate the perception and use of LMS as major tools for formative assessment. Results revealed that professors are generally struggling with the existing tools in LMS and perceive them to be only marginally effective to conduct formative assessment to the extent that they desire, especially at higher order level. They also consider the existing number of tools as being limited. Recommendations were made to improve the design of assessment tools in LMS for formative purposes. Training was identified as the main factor to increase the use of these tools along with receiving institutional support, extra time and technical help in integrating the new tools in their teaching.

Tese de mestrado. Multimédia. Faculdade de Engenharia. Universidade do Porto, Instituto Superior de Engenharia do Porto. 2009

<p>Dagens utveckling inom distansutbildning har lett till att det blir allt viktigare att välja en lärplattform som är anpassat till företagets eller organisationens verksamhet. Nya effektiva utbildningsmetoder som komplement till den traditionella utbildningen möjliggörs av den nya tekniken där e-learning har en framträdande roll. E-learning gör att lärandet inte längre är beroende av ett fysiskt klassrum och en specifik tidpunkt utan utbildningen blir mer flexibel, interaktiv samt fokuserar mer på själva på lärandet.</p><p>Syftet med denna uppsats har varit att kartlägga hur ett LMS bör vara utformat vad det gäller design, funktion och användbarhetsfaktorer för att underlätta val av lärplattform (LMS). Studien har visat att det designmässigt är lämpligt att följa vedertagna författares riktlinjer – oavsett om det gäller LMS eller design av en funktion i ett LMS. När det handlar om användbarhetsfaktorer har studien visat att vissa val av funktioner kan vara avgörande.</p><p>De slutsatser som framkommit i vår studie är att en organisation eller ett företag bör välja ett LMS som är utbyggbart. Efterhand som företaget eller organisationen utvecklas inom sitt LMS, efterfrågas fler funktioner och möjligheter. Det har visat sig att LMS som följer en internationell standard, SCORM – vilket möjliggör återanvändning av material samt överföring mellan olika LMS, är att föredra. </p><p>Vidare visade studien att framtidens LMS måste anpassas till kommande behov både vad det gäller möjligheter för den enskilde användaren att göra personliga inställningar samt att kunna kommunicera via exempelvis chat, forum, e-post, webbkamera eller liknande. Vidare måste fokus ligga på att främja lärandet, att välja ett LMS med potential att växa och personalisera tror vi främjar lärandet i positiv riktning.</p>

Dagens utveckling inom distansutbildning har lett till att det blir allt viktigare att välja en lärplattform som är anpassat till företagets eller organisationens verksamhet. Nya effektiva utbildningsmetoder som komplement till den traditionella utbildningen möjliggörs av den nya tekniken där e-learning har en framträdande roll. E-learning gör att lärandet inte längre är beroende av ett fysiskt klassrum och en specifik tidpunkt utan utbildningen blir mer flexibel, interaktiv samt fokuserar mer på själva på lärandet. Syftet med denna uppsats har varit att kartlägga hur ett LMS bör vara utformat vad det gäller design, funktion och användbarhetsfaktorer för att underlätta val av lärplattform (LMS). Studien har visat att det designmässigt är lämpligt att följa vedertagna författares riktlinjer – oavsett om det gäller LMS eller design av en funktion i ett LMS. När det handlar om användbarhetsfaktorer har studien visat att vissa val av funktioner kan vara avgörande. De slutsatser som framkommit i vår studie är att en organisation eller ett företag bör välja ett LMS som är utbyggbart. Efterhand som företaget eller organisationen utvecklas inom sitt LMS, efterfrågas fler funktioner och möjligheter. Det har visat sig att LMS som följer en internationell standard, SCORM – vilket möjliggör återanvändning av material samt överföring mellan olika LMS, är att föredra. Vidare visade studien att framtidens LMS måste anpassas till kommande behov både vad det gäller möjligheter för den enskilde användaren att göra personliga inställningar samt att kunna kommunicera via exempelvis chat, forum, e-post, webbkamera eller liknande. Vidare måste fokus ligga på att främja lärandet, att välja ett LMS med potential att växa och personalisera tror vi främjar lärandet i positiv riktning.

University of Technology, Sydney. School of Management, Faculty of Business. / This study investigated how selected Australian universities evaluated and adopted various learning management systems in their teaching and learning programs, given claims of uncritical evaluation, problems and cautions in the Australian (1998: 13; Brabazon, 2002; Yetton, Forster, Hewson, Hughes, Johnston, Nightingale, Page-Hanify, Vitale and Wills, 1997) and North American (Berg, 2002; Noble, 1998b) higher education literatures. Ironically, universities charge large amounts of money teaching their students to develop competence in critical analysis, yet some studies have claimed that they were deficient in critically analysing their own decisions (Brabazon, 2002; Yetton et al., 1997). This important question has received little attention in the higher education literature, despite the high visibility and costs of these decisions. Although limited theoretical explanations have been proposed by various researchers, such as Yetton et al. (1997) and Brabazon (2002), these matters have not been the subject of published empirical research to date. A grounded theory methodological framework, validated by the insights of institutional theory, was employed throughout to promote broader sociological explanations than other studies constrained by functionalist theoretical frameworks (Yetton et al., 1997). Qualitative case studies utilising semi-structured interviews and document analysis were conducted at three Australian universities. The findings of this analysis were written up in three case study narratives and an analytic cross-case analysis. Semi-structured interviews and document analysis at the field level were undertaken as an additional source of data to verify emergent grounded theory. A grounded theory of uncritical decision making (Figure 57) was ultimately developed in response to this study’s research problem. The core category around which this model was developed (‘falling behind’) appeared in all three cases, in interviews with experts from the Australian higher education sector, and was also found in both the Australian and overseas higher education literatures. This grounded theory also represents a minor contribution to the institutional theory literature as a new institutional change process model which links the activities of key individuals with broader field developments, and integrates the constructive and reproductive assumptions of old and new institutional theory.

University of Technology, Sydney. School of Management, Faculty of Business. / This study investigated how selected Australian universities evaluated and adopted various learning management systems in their teaching and learning programs, given claims of uncritical evaluation, problems and cautions in the Australian (1998: 13; Brabazon, 2002; Yetton, Forster, Hewson, Hughes, Johnston, Nightingale, Page-Hanify, Vitale and Wills, 1997) and North American (Berg, 2002; Noble, 1998b) higher education literatures. Ironically, universities charge large amounts of money teaching their students to develop competence in critical analysis, yet some studies have claimed that they were deficient in critically analysing their own decisions (Brabazon, 2002; Yetton et al., 1997). This important question has received little attention in the higher education literature, despite the high visibility and costs of these decisions. Although limited theoretical explanations have been proposed by various researchers, such as Yetton et al. (1997) and Brabazon (2002), these matters have not been the subject of published empirical research to date. A grounded theory methodological framework, validated by the insights of institutional theory, was employed throughout to promote broader sociological explanations than other studies constrained by functionalist theoretical frameworks (Yetton et al., 1997). Qualitative case studies utilising semi-structured interviews and document analysis were conducted at three Australian universities. The findings of this analysis were written up in three case study narratives and an analytic cross-case analysis. Semi-structured interviews and document analysis at the field level were undertaken as an additional source of data to verify emergent grounded theory. A grounded theory of uncritical decision making (Figure 57) was ultimately developed in response to this study’s research problem. The core category around which this model was developed (‘falling behind’) appeared in all three cases, in interviews with experts from the Australian higher education sector, and was also found in both the Australian and overseas higher education literatures. This grounded theory also represents a minor contribution to the institutional theory literature as a new institutional change process model which links the activities of key individuals with broader field developments, and integrates the constructive and reproductive assumptions of old and new institutional theory.

University of Technology, Sydney. School of Management, Faculty of Business. / This study investigated how selected Australian universities evaluated and adopted various learning management systems in their teaching and learning programs, given claims of uncritical evaluation, problems and cautions in the Australian (1998: 13; Brabazon, 2002; Yetton, Forster, Hewson, Hughes, Johnston, Nightingale, Page-Hanify, Vitale and Wills, 1997) and North American (Berg, 2002; Noble, 1998b) higher education literatures. Ironically, universities charge large amounts of money teaching their students to develop competence in critical analysis, yet some studies have claimed that they were deficient in critically analysing their own decisions (Brabazon, 2002; Yetton et al., 1997). This important question has received little attention in the higher education literature, despite the high visibility and costs of these decisions. Although limited theoretical explanations have been proposed by various researchers, such as Yetton et al. (1997) and Brabazon (2002), these matters have not been the subject of published empirical research to date. A grounded theory methodological framework, validated by the insights of institutional theory, was employed throughout to promote broader sociological explanations than other studies constrained by functionalist theoretical frameworks (Yetton et al., 1997). Qualitative case studies utilising semi-structured interviews and document analysis were conducted at three Australian universities. The findings of this analysis were written up in three case study narratives and an analytic cross-case analysis. Semi-structured interviews and document analysis at the field level were undertaken as an additional source of data to verify emergent grounded theory. A grounded theory of uncritical decision making (Figure 57) was ultimately developed in response to this study’s research problem. The core category around which this model was developed (‘falling behind’) appeared in all three cases, in interviews with experts from the Australian higher education sector, and was also found in both the Australian and overseas higher education literatures. This grounded theory also represents a minor contribution to the institutional theory literature as a new institutional change process model which links the activities of key individuals with broader field developments, and integrates the constructive and reproductive assumptions of old and new institutional theory.

Volery and Lord noted that the rapid expansion of the Internet will play a pivotal role in the delivery mediums in which education is provided. My dissertation addresses the potential best methodology for developing more effective instruction through Internet based learning. Previous research has identified various aspects related to student learning in online settings, the Internet as a core delivery platform for information, student learning styles, and gender as a component of student success in online platforms. This study was motivated by three research goals: (1) Assess the essential components for an effective open campus model, (2) identify and analyze stakeholder perceptions of factors influencing the design, development, implementation of online coursework, and perceptions by instructional design themes (3) utilize data from literature, respondent questionnaires, agricultural learning modules, and agricultural stakeholder perceptions to propose a model for open online course design and faculty implementation of best teaching methodologies related to online instructional delivery. Using the Delphi Model for consensus, three expert panels (undergraduate agriculture students, secondary agriculture teachers, and post secondary agriculture education faculty) identified the vital components for online course frameworks addressing instructional design, application of course content, and collaborative education models within online learning platforms. To illustrate the objectives of this study, the findings support the proposal and development of the Unified Model for Online Learning and System Design.

This thesis seeks to contribute to the field of learning management system (LMS) development in tertiary educational institutions, particularly to advance the adoption of learning management systems (LMSes) by exploring the incorporation of sociallymotivated discussion forum models. This study proposes a Web-based application, which includes four different discussion forum models for LMSes, in order to test usability and student preferences. The purpose of this study was to compare two non-social discussion forums and two social discussion forums, to determine their appropriateness in terms of attributes or features and general functionality for LMSes. The design processes led to the creation of a Web-based application called 4DFs, which includes four different discussion forum models. Two of these models are non-social discussion forums: the chat room unstructured model and the traditional general threaded discussion. The other two types are social discussion forums, where users can choose who they converse with: the Twitter-style short comment feed and the Facebook-style. The chat room and the traditional general threaded discussion forums' features are based on those of Sakai, since the research sample was comprised of students from the University of Cape Town (UCT). The Twitter-style and Facebook-style elements, such as retweets, hashtags, likes and reposts, are based on Twitter and Facebook. A pilot study was conducted to discover any errors or issues with the experimental procedure. A controlled experiment was then conducted with 31 students from the institution. Participants had to fill out a background information survey to gather some demographic information and to understand more about participants' previous experiences using chat rooms, discussion forums, and social media applications for university related purposes and for non-university related purposes. Following that, participants were given tasks to test all the features of the different discussion forum models. To avoid bias in the participants' choosing of discussion forum models, the research was conducted with a Counterbalanced Measures Design. Participants had to fill in the System Usability Scale (SUS) questionnaire in conjunction with their use of the Web-based application. Then, after using all discussion forums, they had to fill out a preferences questionnaire that asked about their preferences of the discussion forums and the features. The Twitter-style short comment feed model was preferred in terms of the ease of use and since participants were familiar with this forum. This was followed by the chat room unstructured model and the traditional general threaded discussion in terms of these forums' ease of use and students' preference for the layout. The Facebook-style was less preferable. Also, participants indicated that the post button, reply button, edit, delete, and search button were more beneficial features. Participants mention that the layout of the chat room unstructured model was not optimal, since the massive amount of text made it confusing and unclear to decipher. Participants suggested that including the uploading of media, allowing private chat, adding extra features for important posts, and using a repost button in the discussion forums would be more useful. The study found that students preferred that the learning forum include certain characteristics; they prioritised ease of use, less complexity, less interaction and a user-friendly interface over their familiarity with the forum. For learning, there is a need to use the features for a specific purpose so users do not necessarily want extra fancy features (like emojis), instead they want systems that help them to learn efficiently.

The concept of usability refers to a user's capability to interact with a system to fulfill goals in terms of task completion (effectiveness), time measurement (efficiency), and positive attitude (satisfaction). The strategy for usability evaluation in software systems usually involves questionnaires, user testing, and heuristics. Although these methods have been widely used due to several benefits, there are challenges related to time consumption and embedded bias. In response to these challenges, this work proposes a hybrid approach based on usability questionnaire answers and machine learning algorithms to predict usability scores. We describe three different experiments with features extracted from a Learning Management System. These features were applied in the Machine Learning algorithms Linear Regression, Decision Trees, Random Forest, and Neural Networks in three experiments. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements for better performance of the System Usability Scale and UseLearn scores prediction. This approach has potential as a reliable predictive tool for usability scores, which would help create software systems that better satisfy users' needs. / Master of Science / Instructors and students have used online platforms known as Learning Management Systems (LMSs) to improve learning and satisfaction. Students need to achieve their learning goals by interacting with these systems. To achieve these goals, usability evaluation involves ensuring that LMSs attain effectiveness (task completion), efficiency (time measurement), and satisfaction (positive attitude). Usability evaluation usually follows questionnaires, user testing of the LMS, and expert reviews. Although these methods are widely used due to several benefits, they face challenges related to trying these software systems multiple times until the system satisfies student needs and human subjectivity perception. To face these challenges, promote student engagement with the system, and create a better design in the LMS courses, we propose a hybrid approach based on data, questionnaire answers, and machine learning algorithms to predict usability scores. We evaluated this approach through a case study with data collected from undergraduate students at Virginia Tech. The results showed different advantages and drawbacks of machine learning performance. The approach contributes to the engineering and computing education field by providing a reliable predictive tool for usability scores to improve the student learning experience and the features of the LMS.