"Intelligent Tutoring Systems have historically been shown to be an effective means of educating an audience. While there is great benefit from such systems they are generally very costly to build and maintain. It has been estimated that 200 hours of time is required to produce one hour of Intelligent Tutoring System content. The Office of Navel Research has funding this thesis because they are interested in reducing the cost of construction for Intelligent Tutoring Systems. In order for Intelligent Tutoring Systems to be widely accepted and used in the classroom environment there needs to be a toolset that allows for even the most novice user to maintain and grow the system with minimal cost. The goal of this thesis is to create such a toolset targeted towards the Assistments Project. One of the goals of the Assistments Project is to provide a means for teachers to receive meaningful data from the system that they can take to the classroom environment thus enabling a comprehensive learning solution. The effectiveness of the toolset was measured by its ability to reduce the overall time taken to package and distribute content in an Intelligent Tutoring System by providing the tools and allowing the completion of the tasks to be at a reasonable speed."

Intelligent Tutoring Systems are notoriously costly to construct, and require PhD level experience in cognitive science and rule based programming. The purpose of this research was to ease the development process for building pseudo-tutors. Pseudo-tutors are ITS constructs that mimic cognitive tutors but are limited in that they only apply to a single problem. The Assistment Builder is a tool designed to rapidly create, test, and deploy simple pseudo-tutors. These tutors provide a simplified cognitive model based upon a state graph designed for a specific problem. These tutors offer many of the features of rule-based tutors, but with shorter creation time. The system simplifies the process of tutor creation to allow users with little or no ITS experience to develop content. The system provides a web-based interface as a means to build and store these simple tutors we have called Assistments. This paper describes our attempt to make the process of developing, testing, and deploying content easy for teachers. We present data to suggest that users can develop a tutor that can be released to students in approximately an hour.

A major issue in Intelligent Tutoring Systems is off-task student behavior, especially performance-based gaming, where students systematically exploit tutor behavior in order to advance through a curriculum quickly and easily, with as little active thought directed at the educational content as possible. The goal of this research was to explore the phenomena of off-task gaming behavior within the Assistments system, as well as to develop a passive visual indicator to deter and prevent off-task gaming behavior without active intervention via graphical feedback to the student and teachers. Traditional active intervention approaches were also constructed for comparison purposes, and machine-learned gaming-detection models were developed as a potential invocation and evaluation mechanism. Passive graphical interventions have been well received by teachers, and results are suggestive that they are effective at reducing off-task gaming behavior.

Detection and prevention of off-task student behavior in an Intelligent Tutoring System (ITS) has gained a significant amount of attention in recent years. Previous work in these areas have shown some success and improvement. However, the research has largely ignored the incorporation of the expert on student behavior in the classroom: the teacher. Our research re-evaluates the subjects of off-task behavior detection and prevention by developing metrics for student engagement in an ITS using teacher observations of student behavior in the classroom. We present an exploratory analysis of such metrics and the data gathered from the teachers. For off-task prevention we developed a visual reporting tool that displays a representation of a student's activity in an ITS as they progress and gives a valuable immediate report for the instructor.

The Common Tutor Object Platform (CTOP) was designed as a lightweight component framework for creating and deploying applications relating to Intelligent Tutoring Systems. The CTOP supports a runtime for intelligent tutoring system content deployment, a content development environment, an extensive reporting tool, and other smaller applications. The CTOP was designed with future development in mind, allowing easy specification of new base objects and extension points for future development. It has been used as the foundation of the Assistments Project, a wide scale server based ITS deployment. This thesis documents the software engineering aspects of the project. The Assistments Project is capable of supporting a quarter of targeted students in Massachusetts, and optimistically scalable to the entire state and beyond.

The need to give educators the ability to enter a particular teaching strategy of their choice into an Intelligent Tutoring System (ITS) and have the ITS respond appropriately to each student has been stated by many researchers. For example, an educator could tell the ITS to keep students within a certain help level ratio (how much help they request), or to introduce a new topic in a particular manner and the ITS simply carries this out at each learning point of interest. Educators could then try new strategies, ones that unaided are impossible to try out in class (such as keeping a student within a help-seeking range) or difficult within an ITS (as the ITS would have to be specially programmed in that way). Current ITSs provide adaptivity to the student at the domain level but not necessarily at the pedagogical level. While a variety of pedagogical strategies have been implemented (e.g. apprenticeship, socratic, practice), there is no system that offers parts or all of these strategies with the ability to choose between them dynamically. In this project, we designed a new framework for an ITS to include multiple, potentially adaptable pedagogical strategies. This was done by breaking up the pedagogical module into separate components. The Pedagogical Strategy Set (PSS) contains all the strategies, written as constraints. The Pedagogical Student Model (PSM) keeps track of which pedagogical strategies were used by each student. Within the ITS, there is still a smaller, separate pedagogical module to deal with domain-specific strategies. The Pedagogical Control Centre (PCC) contains the logic of when and how to use the pedagogical strategies. It gathers its information from the other modules and uses decision logic to trigger strategies. We implemented and evaluated this framework within the context of SQL-Tutor and found that the framework could be used to enter pedagogical strategies, which in turn compared favourably to the original SQL-Tutor. This proof of concept opens up the possibility of the logic and algorithms that could be implemented (e.g. in the PCC) in future ITSs. The PSS is a separate module, written in a different language, independent of ITSs. This could lead to sharing of pedagogical strategies between tutors. Furthermore, students learn differently to each other; this framework allows them to do so.

The goal of this study was to evaluate whether Electroencephalography (EEG) estimates of attention and cognitive workload captured as students solved math problems could be used to predict success or failure at solving the problems. Students solved a series of SAT math problems while wearing an EEG headset that generated estimates of sustained attention and cognitive workload each second. Students also reported on their level of frustration and the perceived difficulty of each problem. Results from a Support Vector Machine (SVM) training indicated that problem outcomes could be correctly predicted from the combination of attention and workload signals at rates better than chance. The EEG data was also correlated with students' self-report of problem difficulty. Findings suggest that relatively non-intrusive EEG technologies could be used to improve the efficacy of tutoring systems.

The aim of this PhD thesis is to develop a framework for adaptive intelligent tutoring systems (ITS) in the domain of Modern Standard Arabic language. This framework will comprise of a new approach to using a fuzzy inference mechanism and generic rules in guiding the learning process. In addition, the framework will demonstrate another contribution in which the system can be adapted to be used in the teaching of different languages. A prototype system will be developed to demonstrate these features. This system is targeted at adult English-speaking casual learners with no pre-knowledge of the Arabic language. It will consist of two parts: an ITS for learners to use and a teachers‘ tool for configuring and customising the teaching rules and artificial intelligence components among other configuration operations. The system also provides a diverse teaching-strategies‘ environment based on multiple instructional strategies. This approach is based on general rules that provide means to a reconfigurable prediction. The ITS determines the learner‘s learning characteristics using multiple fuzzy inferences. It has a reconfigurable design that can be altered by the teacher at runtime via a teacher-interface. A framework for an independent domain (i.e. pluggable-domain) for foreign language tutoring systems is introduced in this research. This approach allows the system to adapt to the teaching of a different language with little changes required. Such a feature has the advantages of reducing the time and cost required for building intelligent language tutoring systems. To evaluate the proposed system, two experiments are conducted with two versions of the software: the ITS and a cut down version with no artificial intelligence components. The learners used the ITS had shown an increase in scores between the post-test and the pre-test with learning gain of 35% compared to 25% of the learners from the cut down version.

While games can be an innovative and a highly promising approach to education, creating effective educational games is a challenge. It requires effectively integrating educational content with game attributes and aligning cognitive and affective outcomes, which can be in conflict with each other. Intelligent Tutoring Systems (ITS), on the other hand, have proven to be effective learning environments that are conducive to strong learning outcomes. Direct comparisons between tutoring systems and educational games have found digital tutors to be more effective at producing learning gains. However, tutoring systems have had difficulties in maintaining students€™ interest and engagement for long periods of time, which limits their ability to generate learning in the long-term. Given the complementary benefits of games and digital tutors, there has been considerable effort to combine these two fields. This dissertation undertakes and analyzes three different ways of integrating Intelligent Tutoring Systems and digital games. We created three game-like systems with cognition, metacognition and affect as their primary target and mode of intervention. Monkey's Revenge is a game-like math tutor that offers cognitive tutoring in a game-like environment. The Learning Dashboard is a game-like metacognitive support tool for students using Mathspring, an ITS. Mosaic comprises a series of mini-math games that pop-up within Mathspring to enhance students' affect. The methodology consisted of multiple randomized controlled studies ran to evaluate each of these three interventions, attempting to understand their effect on students€™ performance, affect and perception of the intervention and the system that embeds it. Further, we used causal modeling to further explore mechanisms of action, the inter-relationships between student€™s incoming characteristics and predispositions, their mechanisms of interaction with the tutor, and the ultimate learning outcomes and perceptions of the learning experience.

In the field of Artificial Intelligence in Education, many contributions have been made toward estimating student proficiency in Intelligent Tutoring Systems (cf. Corbett & Anderson, 1995). Although the community is increasingly capable of estimating how much a student knows, this does not shed much light on when the knowledge was acquired. In recent research (Baker, Goldstein, & Heffernan, 2010), we created a model that attempts to answer that exact question. We call the model P(J), for the probability that a student just learned from the last problem they answered. We demonstrated an analysis of changes in P(J) that we call “spikiness", defined as the maximum value of P(J) for a student/knowledge component (KC) pair, divided by the average value of P(J) for that same student/KC pair. Spikiness is directly correlated with final student knowledge, meaning that spikes can be an early predictor of success. It has been shown that both over-practice and under-practice can be detrimental to student learning, so using this model can potentially help bias tutors toward ideal practice schedules. After demonstrating the validity of the P(J) model in both CMU's Cognitive Tutor and WPI's ASSISTments Tutoring System, we conducted a pilot study to test the utility of our model. The experiment included a balanced pre/post-test and three conditions for proficiency assessment tested across 6 knowledge components. In the first condition, students are considered to have mastered a KC after correctly answering 3 questions in a row. The second condition uses Bayesian Knowledge Tracing and accepts a student as proficient once they earn a current knowledge probability (Ln) of 0.95 or higher. Finally, we test P(J), which accepts mastery if a student's P(J) value spikes from one problem and the next first response is correct. In this work, we will discuss the details of deriving P(J), our experiment and its results, as well as potential ways this model could be utilized to improve the effectiveness of cognitive mastery learning.