The ASSISTment system is a web-based tutor that is currently being used as an eighth and tenth-grade mathematics in both Massachusetts and Pennsylvania. This system represents its tutors as state-based "pseudo-tutors" which mimic a more complex cognitive tutor based on a set of production rules. It has been shown that building pseudo-tutors significantly decreases the time spent authoring content. This is an advantage for authoring systems such as the ASSITment builder, though it sacrifices greater expressive power and flexibility. A cognitive tutor models a student's behavior with general logical rules. Through model-tracing of a cognitive tutor's rule space, a system can find the reasons behind a student action and give better tutoring. In addition, these cognitive rules are general and can be used for many different tutors. It is the goal of this thesis to provide the architecture for using cognitive rule-based tutors in the ASSITment system. A final requirement is that running these computationally intensive model-tracing tutors do not slow down students using the pseudo-tutors, which represents the majority of ASSISTment usage. This can be achieved with remote computation, realized with SOAP web services. The system was further extended to allow the creation and implementation of user-level experiments within the system. These experiments allow the testing of pedagogical choices. We implemented a hint dissuasion experiment to test this experimental framework and provide those results.

This thesis makes a contribution to Intelligent Tutoring Systems (ITS) architectures. A new intelligent tutoring system is presented for the domain of solving linear equations. This system is novel, because it is the first intelligent equation-solving tutor that combines a cognitive model of the domain with a model of dialog-based tutoring. The tutorial model is novel because it is based on the observation of an experienced human tutor and captures tutorial strategies specific to the domain of equation-solving. In this context, a tutorial dialog is the equivalent of breaking down problems into simpler steps and then asking new questions to the student before proceeding to the next navigational step. The resulting system, named E-tutor, was compared, via a randomized controlled experiment, to an algebra ITS similar to the“Cognitive Tutor" by Carnegie Learning, Inc®. The Cognitive Tutor can provide traditional model-tracing feedback and buggy messages to students, but does not engage students in dialog. Preliminary results using a very small sample size, i.e., teaching equation solving to 15 high school students, showed that E-Tutor with dialog capabilities performed better than E-tutor without dialog. This result showed an effect size of 0.4 standard deviations for overall learning by condition. This set of preliminary results, though not statistically significant, shows promising opportunities to improve learning performance by adding tutorial dialog capabilities to ITSs. However, significant further validation is required, specifically, adding greater numbers and variations of the work to our sample size, before this approach can be deemed successful. The system is available at www.wpi.edu/~leenar/E-tutor.

"Intelligent tutoring systems (ITSs) are computer based tutoring systems that provide individualized tutoring to the students. Building an ITS is recognized to be expensive task in terms of cost and resources. Authoring tools provide a framework and an environment for building the ITSs that help to reduce the resources like skills, time and cost required to build an intelligent tutoring system. In this thesis we have implemented the Cognitive Tutor Authoring Tools (CTAT) and performed experiments to empirically determine the common programming errors that authors tend to make while building an ITS and study what is hard in authoring an ITS. The CTAT were used in a graduate class at Worcester Polytechnic Institute and also at the 4th Summer school organized at the Carnegie Mellon University. Based on the analysis of the experiments we suggest future work to reduce the debugging time and thereby reduce the time required to author an ITS. We also implemented the model tracing algorithm in JESS, evaluated its performance and compared to that of the model tracing algorithm in TDK. This research is funded by the Office of Naval Research (Grant # N00014-0301-0221)."

Cognitive Science is interested in being able to develop methodologies for analyzing human learning and performance data. Intelligent tutoring systems need good cognitive models that can predict student performance. Cognitive models of human processing are also useful in tutoring because well-designed curriculums need to understand the common components of knowledge that students need to be able to employ. A common concern is being able to predict when transfer should happen. We describe a methodology first used by Koedinger that uses empirical data and cognitively principled task analysis to evaluate the fit of cognitive models. This methodology seems particularly useful when you are trying to find evidence for“hidden" knowledge components, which are hard to assess because they are confounded with accessing other knowledge components. We present this methodology as well as an illustration showing how we are trying to use this method to answer an important cognitive science issue.

The purpose of this research was to apply machine learning techniques to automate rule generation in the construction of Intelligent Tutoring Systems. By using a pair of somewhat intelligent iterative-deepening, depth-first searches, we were able to generate production rules from a set of marked examples and domain background knowledge. Such production rules required independent searches for both the“if" and“then" portion of the rule. This automated rule generation allows generalized rules with a small number of sub-operations to be generated in a reasonable amount of time, and provides non-programmer domain experts with a tool for developing Intelligent Tutoring Systems.

The impact of affect on learning and performance has caused many researchers in the field of cognitive psychology to acknowledge the value of motivationally supportive instruction. Goal orientation, which refers to the perceptions and behaviors of the learner in achievement situations, has been the most predominant theory in learning motivation. However, research suggests multiple components are responsible for affecting student cognitive engagement. The traditional framework distinguishes individuals who are self-motivated to master challenging tasks from those who are motivated to earn favorable judgments of performance as intrinsic and extrinsic learners, respectively. In addition, learners may be further categorized by an eagerness to ensure a positive outcome or by their vigilance in avoiding negative outcomes. As such, my research explores how these motivational categories can be utilized to construct a more robust instructional model. The objective of this research is to evaluate the effectiveness of adaptive remediation strategies on motivation and learning performance. Research suggests the cost of integrating cognitive tasks with error analysis outweigh the benefits of sparse learning gains. However, further investigation is required to understand how feedback can improve these outcomes. The experiment presented here seeks to evaluate the adaptive instruction of two pedagogical agents embedded within two separate versions of the Virtual BNI Trainer. The basic coach uses a model of the learner's experience level to determine an appropriate level of elaboration required during remediation. In contrast, the motivationally enhanced coach uses a model of the learner's goal orientation to construct feedback that appeals to their natural disposition. A controlled experiment was conducted to evaluate the effects of adaptive instruction on student self-efficacy, engagement, and learning performance in the Virtual BNI Training Environment. The results of this experiment are used to establish guidelines for integrating goal orientation, error analysis, and feedback within a virtual coach, to improve motivation and learning performance. In addition, these findings also indicate areas for future research.