In order to improve the feedback an intelligent tutoring system provides, the grading engine needs to do more than simply indicate whether a student gives a correct answer or not. Good feedback must provide actionable information with diagnostic value. This means the grading system must be able to determine what knowledge gap or misconception may have caused the student to answer a question incorrectly. This research evaluated the quality of a rules-based grading engine in an automated online homework system by comparing grading engine scores with manually graded scores. The research sought to improve the grading engine by assessing student understanding using knowledge component research. Comparing both the current student scores and the new student scores with the manually graded scores led us to believe the grading engine rules were improved. By better aligning grading engine rules with requisite knowledge components and making revisions to task instructions the quality of the feedback provided would likely be enhanced.

Prerequisite skill structure graphs represent the relationships between knowledge components. Prerequisite structure graphs also propose the order in which students in a given curriculum need to be taught specific knowledge components in order to assist them build on previous knowledge and improve achievement in those subject domains. The importance of accurate prerequisite skill structure graphs can therefore not be overemphasized. In view of this, many approaches have been employed by domain experts to design and implement these prerequisite structures. A number of data mining techniques have also been proposed to infer these knowledge structures from learner performance data. These methods have achieved varied degrees of success. Moreover, to the best of our knowledge, none of the methods have employed extensive randomized controlled trials to learn about prerequisite skill relationships among skills. In this dissertation, we motivate the need for using randomized controlled trials to refine prerequisite skill structure graphs. Additionally, we present PLACEments, an adaptive testing system that uses a prerequisite skill structure graph to identify gaps in students’ knowledge. Students with identified gaps are assisted with more practice assignments to ensure that the gaps are closed. PLACEments additionally allows for randomized controlled experiments to be performed on the underlying prerequisite skill structure graph for the purpose of refining the structure. We present some of the different experiment categories which are possible in PLACEments and report the results of one of these experiment categories. The ultimate goal is to inform domain experts and curriculum designers as they create policies that govern the sequencing and pacing of contents in learning domains whose content lend themselves to sequencing. By extension students and teachers who apply these policies benefit from the findings of these experiments.

Analyzing human learning and performance accurately is one of the main goals of an Intelligent Tutoring System. The“ASSISTment" system is a web-based system that blends assisting students and assessing their performance by providing feedback to the teachers. Good cognitive models are needed for an Intelligent Tutoring system to do a better job at predicting student performance. The ASSISTment system uses a method of cognitive modeling which is called a transfer model. A Transfer Model is a matrix that maps questions to skills. Other researchers have shown that transfer models help in building better predictive models that in-turn help in assessing a student's performance [1, 8]. They provide a viable means of representing a subject matter expert's view of which skills are needed to solve a given problem. However, the process of building a transfer model requires a lot of time. Reducing the time in which a transfer model is built would in turn help reduce the cost of building an Intelligent Tutoring System. Being able to build better transfer models will provide more efficient means of predicting learning in an intelligent tutoring system [6]. In this thesis we studied the creation of one transfer model that maps approximately the 263 released MCAS items to approximately 90 skills. Recently, [5] and [9], using two different modeling methodologies, have both concluded that this transfer model can be used to predict MCAS scores more accurately. Currently the time spent in creating and storing a model is estimated to be approximately 65 hours. This thesis was motivated by the need of a set of tools that would reduce the time spent in building a transfer model. The goal of this thesis was to create a tool that would speed up the process of building a transfer model. The efficiency of this tool is measured by an estimate of the overall time reduced for building a model. The average time reduced by using the tool on per question basis is also measured. The tool is not evaluated for its usability or for the ability to build better fitting transfer models.

Yes / Information systems (IS) facilitate organisations to increase responsiveness and reduce the costs of their supply chain. This paper seeks to make a contribution through exploring and visualising knowledge mapping from the perspective of IS investment evaluation. The evaluation of IS is regarded as a challenging and complex process, which becomes even more difficult with the increased complexity of IS. The intricacy of IS evaluation, however, is due to numerous interrelated factors (e.g. costs, benefits and risks) that have human or organisational dimensions. With this in mind, there appears to be an increasing need to assess investment decision-making processes, to better understand the often far-reaching implications associated with technology adoption and interrelated knowledge components (KC). Through the identification and extrapolation of key learning issues from the literature and empirical findings, organisations can better improve their business processes and thereby their effectiveness and efficiency, while preventing others from making costly oversights that may not necessarily be only financial. In seeking to enlighten the often obscure evaluation of IS investments, this paper attempts to inductively emphasise the dissemination of knowledge and learning through the application of a fuzzy Expert System (ES) based knowledge mapping technique (i.e. Fuzzy Cognitive Map [FCM]). The rationale for exploring knowledge and IS investment evaluation is that a knowledge map will materialise for others to exploit during their specific technology evaluation. This is realised through conceptualising the explicit and tacit investment drivers. Among the several findings drawn from this research, the key resulting knowledge mapping through FCM demonstrated the complex, multifaceted and emergent behaviour of causal relationships within the knowledge area. The principal relationships and knowledge within IS investment evaluation are illustrated as being determined by a blend of managerial and user perspectives.