"Much work has been done to develop intelligent tutoring systems in domains such as algebra, geometry, and computer programming. Our work is to develop an intelligent tutoring system to train US soldiers. One main difference in this domain is that one of the main skills to be learned is cooperation between teammates, so the tutor must emphasize collaboration as a skill. In addition, to help train this skill the system must be able to run in real-time, and provide both computer generated teammates, as well as intelligent opposing forces. This system is the first real-time, multi-user, model tracing tutor with simulated teammates. The goal of this thesis is to build a prototype system to validate that this is a valid approach for this domain."

This thesis makes a contribution to Intelligent Tutoring Systems (ITS) architectures. A new intelligent tutoring system is presented for the domain of solving linear equations. This system is novel, because it is the first intelligent equation-solving tutor that combines a cognitive model of the domain with a model of dialog-based tutoring. The tutorial model is novel because it is based on the observation of an experienced human tutor and captures tutorial strategies specific to the domain of equation-solving. In this context, a tutorial dialog is the equivalent of breaking down problems into simpler steps and then asking new questions to the student before proceeding to the next navigational step. The resulting system, named E-tutor, was compared, via a randomized controlled experiment, to an algebra ITS similar to the“Cognitive Tutor" by Carnegie Learning, Inc®. The Cognitive Tutor can provide traditional model-tracing feedback and buggy messages to students, but does not engage students in dialog. Preliminary results using a very small sample size, i.e., teaching equation solving to 15 high school students, showed that E-Tutor with dialog capabilities performed better than E-tutor without dialog. This result showed an effect size of 0.4 standard deviations for overall learning by condition. This set of preliminary results, though not statistically significant, shows promising opportunities to improve learning performance by adding tutorial dialog capabilities to ITSs. However, significant further validation is required, specifically, adding greater numbers and variations of the work to our sample size, before this approach can be deemed successful. The system is available at www.wpi.edu/~leenar/E-tutor.

"Intelligent tutoring systems (ITSs) are computer based tutoring systems that provide individualized tutoring to the students. Building an ITS is recognized to be expensive task in terms of cost and resources. Authoring tools provide a framework and an environment for building the ITSs that help to reduce the resources like skills, time and cost required to build an intelligent tutoring system. In this thesis we have implemented the Cognitive Tutor Authoring Tools (CTAT) and performed experiments to empirically determine the common programming errors that authors tend to make while building an ITS and study what is hard in authoring an ITS. The CTAT were used in a graduate class at Worcester Polytechnic Institute and also at the 4th Summer school organized at the Carnegie Mellon University. Based on the analysis of the experiments we suggest future work to reduce the debugging time and thereby reduce the time required to author an ITS. We also implemented the model tracing algorithm in JESS, evaluated its performance and compared to that of the model tracing algorithm in TDK. This research is funded by the Office of Naval Research (Grant # N00014-0301-0221)."

"Intelligent Tutoring Systems (ITSs), while effective in enhancing students’ problem solving skills, are difficult and time-consuming to build. In order to reduce the length and the complexity of ITS construction, authoring tools are used. These tools provide a solid foundation for creating pedagogical exercises for students, and offer graphical user interfaces that eliminate the need for programming expertise. One of the major problems with today’s authoring tools is that they are still quite intricate and time-consuming to utilize, even for users who are familiar with them. Their steep learning curves often intimidate users who are only interested in creating simple tutoring systems. I have designed and implemented an authoring tool, called Mason, which strips away the visual interface design features of today’s top ITSs, and focuses on the creation of sophisticated pedagogical exercises using a hierarchical domain model. The exercise creation process includes the definition of numerous components, such as: a problem statement, the desired answer to the exercise, the strategies for tutoring students on the mistakes they make while trying to formulate the correct answer, and diagnostic rules for launching the appropriate strategies for specific student errors. The ultimate goal of Mason is to be able to significantly reduce the time needed to author text-based ITSs that are able to diagnose student answers and generate pedagogical dialogue accordingly. This goal was verified by using Mason to replicate the architecture of Ms. Lindquist, a sophisticated ITS for algebra that originally took over a year a construct. The replica was finished in less than a week, and was able to emulate Ms. Lindquist’s dialogue generation accurately with minor limitations."

The present research incorporated the content of basic statistics into the Artificial Intelligence Physics Tutor (ARPHY), which was used as the expert system shell, and investigated the effects of the Artificial Intelligent Statistics Tutor (ARSTAT) as a supplement to learning statistics at the college level. Two classes of an introductory educational statistics course in the Department of Educational Foundations, University of North Texas, were used in the study. The daytime class was used as the experimental group and the evening class was used as the control group. The experimental group's lecture/discussion was supplemented with ARSTAT, and the control group received only lecture/discussion. A one-way analysis of covariance was used to compare students' test scores. No significant difference was found; however, the adjusted mean score of the experimental group was slightly higher than that of the control group. A two-way analysis of covariance showed no significant main effect or interaction between gender and study technique. A second two-way analysis of covariance showed no significant interaction between the students' attitude toward statistics and the study technique used. However, the students with a statistics-positive attitude scored significantly higher on the test than students who had a negative attitude toward statistics. This study concluded that the ARSTAT can be used effectively as a tutor for students taking an introductory course in educational statistics. The following recommendations for further study were made: incorporate more advanced topics of statistics into the ARPHY teaching model; incorporate the ARPHY learning theory and statistical content using another version of LISP language or another programming language such as PROLOG; and compare the ARSTAT tutor to some other kind of supplement to lecture/discussion.

Cognitive Science is interested in being able to develop methodologies for analyzing human learning and performance data. Intelligent tutoring systems need good cognitive models that can predict student performance. Cognitive models of human processing are also useful in tutoring because well-designed curriculums need to understand the common components of knowledge that students need to be able to employ. A common concern is being able to predict when transfer should happen. We describe a methodology first used by Koedinger that uses empirical data and cognitively principled task analysis to evaluate the fit of cognitive models. This methodology seems particularly useful when you are trying to find evidence for“hidden" knowledge components, which are hard to assess because they are confounded with accessing other knowledge components. We present this methodology as well as an illustration showing how we are trying to use this method to answer an important cognitive science issue.

The purpose of this research was to apply machine learning techniques to automate rule generation in the construction of Intelligent Tutoring Systems. By using a pair of somewhat intelligent iterative-deepening, depth-first searches, we were able to generate production rules from a set of marked examples and domain background knowledge. Such production rules required independent searches for both the“if" and“then" portion of the rule. This automated rule generation allows generalized rules with a small number of sub-operations to be generated in a reasonable amount of time, and provides non-programmer domain experts with a tool for developing Intelligent Tutoring Systems.

Intelligent tutoring systems (ITSs) are computer programs that model individual learners and adapt instruction to help each learner differently. One way ITSs differ from human tutors is that few ITSs give learners a way to ask questions. When learners can ask for help, their questions have the potential to improve learning directly and also act as a new source of model data to help the ITS personalize instruction. Inquiry modeling gives ITSs the ability to answer learner questions and refine their learner models with an inexpensive new input channel. In order to support inquiry modeling, an advanced planning formalism is applied to ITS learner modeling. Partially observable Markov decision processes (POMDPs) differ from more widely used ITS architectures because they can plan complex action sequences in uncertain situations with machine learning. Tractability issues have previously precluded POMDP use in ITS models. This dissertation introduces two improvements, priority queues and observation chains, to make POMDPs scale well and encompass the large problem sizes that real-world ITSs must confront. A new ITS was created to support trainees practicing a military task in a virtual environment. The development of the Inquiry Modeling POMDP Adaptive Trainer (IMP) began with multiple formative studies on human and simulated learners that explored inquiry modeling and POMDPs in intelligent tutoring. The studies suggest the new POMDP representations will be effective in ITS domains having certain common characteristics. iv Finally, a summative study evaluated IMP’s ability to train volunteers in specific practice scenarios. IMP users achieved post-training scores averaging up to 4.5 times higher than users who practiced without support and up to twice as high as trainees who used an ablated version of IMP with no inquiry modeling. IMP’s implementation and evaluation helped explore questions about how inquiry modeling and POMDP ITSs work, while empirically demonstrating their efficacy

This research focused on a formal (theory based) approach to designing Intelligent Tutoring System (ITS) authoring tool involving two specific conventional pedagogical theories—Conversation Theory (CT) and Cognitive Apprenticeship (CA). The research conceptualised an Augmented Conversation and Cognitive Apprenticeship Metamodel (ACCAM) based on apriori theoretical knowledge and assumptions of its underlying theories. ACCAM was implemented in an Intelligent Learning Activity Builder System (ILABS)—an ITS authoring tool. ACCAM’s implementation aims to facilitate formally designed tutoring systems, hence, ILABS―the practical implementation of ACCAM― constructs metamodels for Intelligent Learning Activity Tools (ILATs) in a numerical problem-solving context (focusing on the construction of procedural knowledge in applied numerical disciplines). Also, an Intelligent Learning Activity Management System (ILAMS), although not the focus of this research, was developed as a launchpad for ILATs constructed and to administer learning activities. Hence, ACCAM and ILABS constitute the conceptual and practical contributions that respectively flow from this research. ACCAM’s implementation was tested through the evaluation of ILABS and ILATs within an applied numerical domain―the accounting domain. The evaluation focused on the key constructs of ACCAM―cognitive visibility and conversation, implemented through a tutoring strategy employing Process Monitoring (PM). PM augments conversation within a cognitive apprenticeship framework; it aims to improve the visibility of the cognitive process of a learner and infers intelligence in tutoring systems. PM was implemented via an interface that attempts to bring learner’s thought process to the surface. This approach contrasted with previous studies that adopted standard Artificial Intelligence (AI) based inference techniques. The interface-based PM extends the existing CT and CA work. The strategy (i.e. interface-based PM) makes available a new tutoring approach that aimed fine-grain (or step-wise) feedbacks, unlike the goal-oriented feedbacks of model-tracing. The impact of PM—as a preventive strategy (or intervention) and to aid diagnosis of learners’ cognitive process—was investigated in relation to other constructs from the literature (such as detection of misconception, feedback generation and perceived learning effectiveness). Thus, the conceptualisation and implementation of PM via an interface also contributes to knowledge and practice. The evaluation of the ACCAM-based design approach and investigation of the above mentioned constructs were undertaken through users’ reaction/perception to ILABS and ILAT. This involved, principally, quantitative approach. However, a qualitative approach was also utilised to gain deeper insight. Findings from the evaluation supports the formal (theory based) design approach—the design of ILABS through interaction with ACCAM. Empirical data revealed the presence of conversation and cognitive visibility constructs in ILATs, which were determined through its behaviour during the learning process. This research identified some other theoretical elements (e.g. motivation, reflection, remediation, evaluation, etc.) that possibly play out in a learning process. This clarifies key conceptual variables that should be considered when constructing tutoring systems for applied numerical disciplines (e.g. accounting, engineering). Also, the research revealed that PM enhances the detection of a learner’s misconception and feedback generation. Nevertheless, qualitative data revealed that frequent feedbacks due to the implementation of PM could be obstructive to thought process at advance stage of learning. Thus, PM implementations should also include delayed diagnosis, especially for advance learners who prefer to have it on request. Despite that, current implementation allows users to turn PM off, thereby using alternative learning route. Overall, the research revealed that the implementation of interface-based PM (i.e. conversation and cognitive visibility) improved the visibility of learner’s cognitive process, and this in turn enhanced learning—as perceived.

Adaptive tutorial dialogues have been successfully employed by ITSs to facilitate deep learning of conceptual domain knowledge. But none of the approaches used for generating dialogues have been used across instructional domains and tasks. The objective of this project was twofold: (i) to propose a general model that provides adaptive dialogue support in both well- and ill-defined instructional tasks (ii) to explore whether adaptive tutorial dialogues are better than non-adaptive dialogues in acquiring domain knowledge. Our model provides adaptive dialogue support by identifying the concepts that the student has most difficulty with, and then selecting the tutorial dialogues corresponding to those concepts. The dialogues are customised based on the student’s knowledge and explanation skills, in terms of the length and the exact content of the dialogue. The model consists of three parts: an error hierarchy, tutorial dialogues and rules for adapting them. We incorporated our model into EER-Tutor, a constraint-based tutor that teaches database design. The effectiveness of adaptive dialogues compared to non-adaptive dialogues in learning this ill-defined task was evaluated in an authentic classroom environment. The results revealed that the acquisition of the domain knowledge (represented as constraints) of the experimental group who received adaptive dialogues was significantly higher than their peers in the control group with non-adaptive dialogues. We also incorporated our model into NORMIT, a constraint-based tutor that teaches data normalization. We repeated the experiment using NORMIT in a real-world class room environment with a much smaller group of students (18 in NORMIT study vs 65 in EER-Tutor study) but did not find significant differences. We also investigated whether our model could support dialogues in logical database design and fraction addition using paper-based methods. Our evaluation studies and investigations on paper indicated that our model can provide adaptive support for both ill-and well-defined tasks associated with a well-defined domain theory. The results also indicated that adaptive dialogues are more effective than non-adaptive dialogues in teaching the ill-defined task of database design.