The desire to follow student learning within intelligent tutoring systems in near real time has led to the development of several models anticipating the correctness of the next item as students work through an assignment. Such models have in- cluded Bayesian Knowledge Tracing (BKT), Performance Factors Analysis (PFA), and more recently with developments in Deep Learning, Deep Knowledge Tracing (DKT). The DKT model, based on the use of a recurrent neural network, exhibited promising results in paper [PBH+15]. Thus far, however, the model has only considered the knowledge components of the problems and correctness as input, neglecting the breadth of other features col- lected by computer-based learning platforms. This work seeks to improve upon the DKT model by incorporating more features at the problem-level and student-level. With this higher dimensional input, an adaption to the original DKT model struc- ture is also proposed, incorporating an Autoencoder network layer to convert the input into a low dimensional feature vector to reduce both the resource requirement and time needed to train. Experimental results show that our adapted DKT model, which includes more combinations of features, can effectively improve accuracy.

In the field of Artificial Intelligence in Education, many contributions have been made toward estimating student proficiency in Intelligent Tutoring Systems (cf. Corbett & Anderson, 1995). Although the community is increasingly capable of estimating how much a student knows, this does not shed much light on when the knowledge was acquired. In recent research (Baker, Goldstein, & Heffernan, 2010), we created a model that attempts to answer that exact question. We call the model P(J), for the probability that a student just learned from the last problem they answered. We demonstrated an analysis of changes in P(J) that we call “spikiness", defined as the maximum value of P(J) for a student/knowledge component (KC) pair, divided by the average value of P(J) for that same student/KC pair. Spikiness is directly correlated with final student knowledge, meaning that spikes can be an early predictor of success. It has been shown that both over-practice and under-practice can be detrimental to student learning, so using this model can potentially help bias tutors toward ideal practice schedules. After demonstrating the validity of the P(J) model in both CMU's Cognitive Tutor and WPI's ASSISTments Tutoring System, we conducted a pilot study to test the utility of our model. The experiment included a balanced pre/post-test and three conditions for proficiency assessment tested across 6 knowledge components. In the first condition, students are considered to have mastered a KC after correctly answering 3 questions in a row. The second condition uses Bayesian Knowledge Tracing and accepts a student as proficient once they earn a current knowledge probability (Ln) of 0.95 or higher. Finally, we test P(J), which accepts mastery if a student's P(J) value spikes from one problem and the next first response is correct. In this work, we will discuss the details of deriving P(J), our experiment and its results, as well as potential ways this model could be utilized to improve the effectiveness of cognitive mastery learning.

Predicting student performance is an important part of the student modeling task in Intelligent Tutoring System (ITS). The state-of-art model for predicting student performance - Bayesian Knowledge Tracing (KT) has many critical limitations. One specific limitation is that KT has no underlying mechanism for memory decay represented in the model, which means that no forgetting is happening in the learning process. In addition we notice that numerous modification to the KT model have been proposed and evaluated, however many of these are often based on a combination of intuition and experience in the domain, leading to models without performance improvement. Moreover, KT is computationally expensive, model fitting procedures can take hours or days to run on large datasets. The goal of this research work is to improve the accuracy of student performance prediction by incorporating the memory decay factor which the standard Bayesian Knowledge Tracing had ignored. We also propose a completely data driven and inexpensive approach to model improvement. This alternative allows for researchers to evaluate which aspects of a model are most likely to result in model performance improvements based purely on the dataset features that are computed from ITS system logs.

Student gaming behavior in intelligent tutoring systems (ITS) has been correlated with lower learning rates. The goal of this work is to identify such behavior, produce interventions to discourage this behavior, and by doing so hopefully improve the learning rate of students who would normally display gaming behavior. Detectors have been built to identify gaming behavior. Interventions have been designed to discourage the behavior and their evaluation is discussed.

Over the past couple decades, intelligent tutoring systems (ITSs) have become popular in education. ITSs are effective at helping students learn (VanLehn, 2011; Razzaq, Mendicino & Heffernan, 2008; Koedinger et al, 1997) and help researchers understand how students learn. Such research has included modeling how students learn (Corbett & Anderson, 1995), the effectiveness of help given within an ITS (Beck et al, 2008), the difficulty of different problems (Pardos & Heffernan, 2011), and predicting long-term outcomes like college attendance (San Pedro et al, 2013a), among many other studies. While most studies have focused on ITSs from a cognitive perspective, a growing number of researchers are paying attention to the motivational and affective aspects of tutoring, which have been recognized as important components of human tutoring (Lepper et al, 1993). Recent work has shown that student affect within an ITS can be detected, even without physical sensors or cameras (D’Mello et al, 2008; Conati & Maclaren, 2009; Sabourin et al, 2011; San Pedro et al, 2013b). Initial studies with these sensor-less affect detectors have shown that certain problematic affective states, such as boredom, confusion and frustration, are prevalent within ITSs (Baker et al, 2010b). Boredom in particular has been linked to negative learning outcomes (Pekrun et al, 2010; Farmer & Sundberg, 1986) and long-term disengagement (Farrell, 1988). Therefore, reducing or responding effectively to these affective states within ITSs may improve both short- and long-term learning outcomes. This work is an initial attempt to determine what causes boredom in ITSs. First, we determine which is more responsible for boredom in ITSs: the content in the system, or the students themselves. Based on the findings of that analysis, we conduct a randomized controlled trial to determine the effects of monotony on student boredom. In addition to the work on boredom, we also perform analyses that concern student modeling, specifically how to improve Knowledge Tracing (Corbett & Anderson, 1995), a popular student model used extensively in real systems like the Cognitive Tutors (Koedinger et al, 1997) and in educational research.

Internet-based and mobile technologies enable new ways of learning. They offer us new possibilities to access an enormous amount of knowledge at any time and everywhere. Among many advantages, the adaptations require a rethinking of our previous learning behaviour patterns and processes. The challenge for students is no longer to get access to information and knowledge, but to select the right one and to deal with the information and knowledge overflow. The aim of this research is to define, design and validate an advanced concept to support the contemporary learning processes. Therefore, the requirements for a new approach have been assessed, the available solutions from the related area of (personal) Knowledge Management have been investigated, and the weaknesses in the context of learning identified. The identified issues have been substantiated by university students via a quantitative survey. Besides several smaller aspects, knowledge fragmentation and the nescience of the knowledge formation process have been classified as the most critical ones. To overcome these problems, a methodological concept has been developed, and a corresponding technological design created. The chosen approach is an intelligent, independent intermediate layer, which traces the different steps our knowledge entities are going through. Based on personal and individual configurations, the system provides a comprehensive and overall observation of nearly all our knowledge work activities. It supports the building and accessing of the knowledge formation paths for every important knowledge unit, later path combination and the access to automatically generated versions of our work. Moreover, it helps the users not only to remember what they did, but also gives them some strong indications why they did it. This is achieved by combining different knowledge actions and looking at the influences they have on each other. The suggested concept has been critically proved and confirmed via a qualitative expert analysis and backed up by a quantitative survey among university students.

Two of the major goals in Educational Data Mining are determining students’ state of knowledge and determining their affective state. It is useful to be able to determine whether a student is engaged with a tutor or task in order to adapt to his/her needs and necessary to have an idea of the students' knowledge state in order to provide material that is appropriately challenging. These two problems are usually examined separately and multiple methods have been proposed to solve each of them. However, little work has been done on examining both of these states in parallel and the combined effect on a student’s performance. The work reported in this thesis explores ways to observe both behavior and performance in order to more fully understand student state.

abstract: Embedded assessment constantly updates a model of the student as the student works on instructional tasks. Accurate embedded assessment allows students, instructors and instructional systems to make informed decisions without requiring the student to stop instruction and take a test. This thesis describes the development and comparison of several student models for Dragoon, an intelligent tutoring system. All the models were instances of Bayesian Knowledge Tracing, a standard method. Several methods of parameterization and calibration were explored using two recently developed toolkits, FAST and BNT-SM that replaces constant-valued parameters with logistic regressions. The evaluation was done by calculating the fit of the models to data from human subjects and by assessing the accuracy of their assessment of simulated students. The student models created using node properties as subskills were superior to coarse-grained, skill-only models. Adding this extra level of representation to emission parameters was superior to adding it to transmission parameters. Adding difficulty parameters did not improve fit, contrary to standard practice in psychometrics. / Dissertation/Thesis / Masters Thesis Computer Science 2015

abstract: EMBRACE (Enhanced Moved By Reading to Accelerate Comprehension in English) is an IPad application that uses the Moved By Reading strategy to help improve the reading comprehension skills of bilingual (Spanish speaking) English Language Learners (ELLs). In EMBRACE, students read the text of a story and then move images corresponding to the text that they read. According to the embodied cognition theory, this grounds reading comprehension in physical experiences and thus is more engaging. In this thesis, I used the log data from 20 students in grades 2-5 to design a skill model for a student using EMBRACE. A skill model is the set of knowledge components that a student needs to master in order to comprehend the text in EMBRACE. A good skill model will improve understanding of the mistakes students make and thus aid in the design of useful feedback for the student.. In this context, the skill model consists of vocabulary and syntax associated with the steps that students performed. I mapped each step in EMBRACE to one or more skills (vocabulary and syntax) from the model. After every step, the skill level is updated in the model. Thus, if a student answered the previous step incorrectly, the corresponding skills are decremented and if the student answered the previous question correctly, the corresponding skills are incremented, through the Bayesian Knowledge Tracing algorithm. I then correlated the students’ predicted scores (computed from their skill levels) to their posttest scores. I evaluated the students’ predicted scores (computed from their skill levels) by comparing them to their posttest scores. The two sets of scores were not highly correlated, but the results gave insights into potential improvements that could be made to the system with respect to user interaction, posttest scores and modeling algorithm. / Dissertation/Thesis / Masters Thesis Computer Science 2016

<div>Today, machine learning models and Deep Neural Networks (DNNs) are prevalent in various areas. Also, educational Artificial Intelligence (AI) is drawing increasing attention</div><div>with the rapid development of online learning platforms. Researchers explore different types of educational AI to improve students’ learning performance and experience in online classes. Educational AIs can be categorized into “interactive” and “predictive.” Interactive AIs answer simple course questions for students, such as the due day of homework and the final project’s minimum page requirement. Predictive educational AIs play a role in predicting students’ learning states. Instructors can adjust the learning content based on the students’ learning states. However, most AIs are not evaluated in an actual class setting. Therefore, we want to evaluate the effects of a state-of-the-art educational AI model, BKT (Bayesian Knowledge Tracing)-LSTM(Long Short-Term Memory), on students’ learning performance in an actual class setting. Data came from the course CNIT 25501, a large introductory Java program?ming class at Purdue University. Participants were randomly separated into the control and experimental groups (AI-group). Weekly quizzes measured participants’ learning performance. Pre-quiz and base quizzes estimated participants’ prior knowledge levels. Using BKT-LSTM, participants in the experimental group had questions from the knowledge that they were most lacking. However, participants in the control group had questions from randomly picked knowledge. The results suggested that both the experimental and control groups had lower scores in review quizzes than in base quizzes. However, the score difference between base quizzes and review quizzes for the experimental group was more often significantly different (three quizzes) compared to the control group (two quizzes), demonstrating the predictive capability of BKT-LSTM to some extent. Initially, we expected that BKT-LSTM would enhance students’ learning performance. However, in post-quiz, participants in the control group had significantly higher scores than those in the experimental group. The result suggested that continuous complex questions may negatively affect students’ learning initiatives. On the contrary, relatively easy questions may improve their learning initiatives.</div>