Intelligent Tutoring Systems (ITSs) have been shown to significantly improve students' learning in a variety of domains, including physics, mathematics, and thermodynamics. Tutorial dialogues is one of the strategies used by ITSs and has been empirically shown to significantly improve learning. This project investigates how different students interact with the tutorial dialogues in EER-Tutor, using both eye-gaze data and student-system interaction logs. EER-Tutor is a constraint-based ITS that teaches conceptual database design. In order to have a more comprehensive and accurate picture of a user's interactions with a learning environment, we need to know which interface features s/he visually inspected, what strategies s/he used and what cognitive efforts s/he made to complete tasks. Such knowledge allows intelligent systems to be proactive, rather than reactive, to users' actions. Eye-movement tracking is therefore a potential source of real-time adaptation in a learning environment. Our findings indicate that advanced students are selective of the interface areas they visually focus on whereas novices waste time by paying attention to interface areas that are inappropriate for the task at hand. Novices are also unaware that they need help with understanding the domain concepts discussed in the tutorial dialogues. We were able to accurately classify students, for example as novice or advanced students, using only eye-gaze or EER-Tutor log data as well as a combination of EER-Tutor and eye-gaze features. The cost of eye-tracking is justified as classifiers using only eye-gaze features sometimes perform as well as those utilising both EER-Tutor and eye-gaze data and outperform classifiers using only EER-Tutor data. The ability to classify students will therefore allow an ITS to intervene when needed and better guide students' learning if it detects sub-optimal behaviour.

As one of the central problems in the area of Intelligent Tutoring Systems (ITSs), student modelling has been widely used to assist in systems’ decision making and students’ learning. On the one hand, by reasoning about students’ knowledge in the instructional domain, a system is able to adapt its pedagogical actions in order to provide a customized learning environment. These actions may include individualized problem-selection, tailored instructions and feedback, as well as updating the presentation of student models. On the other hand, students can reflect on their own learning progress by viewing individual Open Student Models (OSMs) and enhance their meta-cognitive skills by learning from the system’s estimation of their knowledge levels. It is believed that making the information in the student model available to students can raise students’ awareness of their strengths and weaknesses in the corresponding domain and hence allow them to develop a more effective and efficient way of learning. An OSM has been developed in EER-Tutor. EER-Tutor is a web-enhanced ITS that supports university students in learning conceptual database modelling. Students design Enhanced Entity-Relationship (EER) diagrams and receive different level of feedback in a problem-solving environment. The pedagogical decisions on feedback generation and problem selection are made according to student models. Previously, student models in EER-Tutor are presented to students on request as skill meters. Skill meters have been proved useful in helping students to improve their meta-cognitive skills. However, as the simplest presentation of a student model, skill meters contain very limited information. Some studies show that an OSM with multiple views is more effective since it supports individual preferences and different educational purposes. The focus of our work is to develop a variety of presentations for the OSM in EER-Tutor. For this purposes, we have modified the system to include not only skill meters but also other presentation styles. An evaluation study has been performed after the development. Both subjective and objective results have been collected. This thesis presents the extended EER-Tutor, followed by the analysis of the evaluation study.

No description available.

No description available.

"Secondary teachers across the United States are being asked to use formative assessment data to inform their classroom instruction. At the same time, critics of US government’s No Child Left Behind legislation are calling the bill “No Child Left Untested”. Among other things, critics point out that every hour spent assessing students is an hour lost from instruction. But, does it have to be? What if we better integrated assessment into classroom instruction and allowed students to learn during the test? This dissertation emphasizes using the intelligent tutoring system as an assessment system that just so happens to provide instructional assistance during the test. Usually it is believed that assessment get harder if students are allowed to learn during the test, as it’s then like trying to hit a moving target. So, my results are somewhat shocking that by providing tutoring to students while they are assessed I actually improve the assessment of students’ knowledge. Most traditional assessments treat all questions on the test as sampling a single underlying knowledge component. Yet, teachers want detailed, diagnostic reports to inform their instruction. Can we have our cake and eat it, too? In this dissertation, I provide solid evidence that a fine-grained skill model is able to predict state test scores better than coarser-rained models, as well as being used to give teachers more informative feedback that they can reflect on to improve their instruction. The contribution of the dissertation lies in that it established novel assessment methods to better assess students in intelligent tutoring systems. Through analyzing data of more than 1,000 students across two years, it provides strong evidence implying that it is possible to develop a continuous assessment system that can do all three of these things at the same time: 1) accurately and longitudinally assesses students, 2) gives fine grained feedback that is more cognitively diagnostic, and 3) saves classroom instruction time by assessing students while they are getting tutoring. "

"Intelligent Tutoring Systems have historically been shown to be an effective means of educating an audience. While there is great benefit from such systems they are generally very costly to build and maintain. It has been estimated that 200 hours of time is required to produce one hour of Intelligent Tutoring System content. The Office of Navel Research has funding this thesis because they are interested in reducing the cost of construction for Intelligent Tutoring Systems. In order for Intelligent Tutoring Systems to be widely accepted and used in the classroom environment there needs to be a toolset that allows for even the most novice user to maintain and grow the system with minimal cost. The goal of this thesis is to create such a toolset targeted towards the Assistments Project. One of the goals of the Assistments Project is to provide a means for teachers to receive meaningful data from the system that they can take to the classroom environment thus enabling a comprehensive learning solution. The effectiveness of the toolset was measured by its ability to reduce the overall time taken to package and distribute content in an Intelligent Tutoring System by providing the tools and allowing the completion of the tasks to be at a reasonable speed."

Two of the major goals in Educational Data Mining are determining students’ state of knowledge and determining their affective state. It is useful to be able to determine whether a student is engaged with a tutor or task in order to adapt to his/her needs and necessary to have an idea of the students' knowledge state in order to provide material that is appropriately challenging. These two problems are usually examined separately and multiple methods have been proposed to solve each of them. However, little work has been done on examining both of these states in parallel and the combined effect on a student’s performance. The work reported in this thesis explores ways to observe both behavior and performance in order to more fully understand student state.

Numerous methods to assess student knowledge are present throughout every step of a students€™ education. Skill-based assessments include homework, quizzes and tests while curriculum exams comprise of the SAT and GRE. The latter assessments provide an indication as to how well a student has retained a learned national curriculum however they are unable to identify how well a student performs at a fine grain skill level. The former assessments hone in on a specific skill or set of skills, however, they require an excessive amount of time to collect curriculum-wide data. We've developed a system that assesses students at a fine grain level in order to identify non- mastered skills within each student€™s zone of proximal development. €œPLACEments€� is a graph-driven computer adaptive test which not only provides thorough student feedback to educators but also delivers a personalized remediation plan to each student based on his or her identified non-mastered skills. As opposed to predicting state test scores, PLACEments objective is to personalize learning for students and encourage teachers to employ formative assessment techniques in the classroom. We have conducted a randomized controlled study to evaluate the learning value PLACEments provides in comparison to traditional methods of targeted skill mastery and retention.

Over the last two decades, the field of educational data mining (EDM) has been focusing on predicting the correctness of the next student response to the question (e.g., [2, 6] and the 2010 KDD Cup), in other words, predicting student short-term performance. Student modeling has been widely used for making such inferences. Although performing well on the immediate next problem is an indicator of mastery, it is by far not the only criteria. For example, the Pittsburgh Science of Learning Center's theoretic framework focuses on robust learning (e.g., [7, 10]), which includes the ability to transfer knowledge to new contexts, preparation for future learning of related skills, and retention - the ability of students to remember the knowledge they learned over a long time period. Especially for a cumulative subject such as mathematics, robust learning, particularly retention, is more important than short-term indicators of mastery. The Automatic Reassessment and Relearning System (ARRS) is a platform we developed and deployed on September 1st, 2012, which is mainly used by middle-school math teachers and their students. This system can help students better retain knowledge through automatically assigning tests to students, giving students opportunity to relearn the skill when necessary and generating reports to teachers. After we deployed and tested the system for about seven months, we have collected 287,424 data points from 6,292 students. We have created several models that predict students' retention performance using a variety of features, and discovered which were important for predicting correctness on a delayed test. We found that the strongest predictor of retention was a student's initial speed of mastering the content. The most striking finding was that students who struggled to master the content (took over 8 practice attempts) showed very poor retention, only 55% correct, after just one week. Our results will help us advance our understanding of learning and potentially improve ITS.

Schools use commercial systems specifically for mathematics benchmarking and longitudinal assessment. However these systems are expensive and their results often fail to indicate a clear path for teachers to differentiate instruction based on students’ individual strengths and weaknesses in specific skills. ASSISTments is a web-based Intelligent Tutoring System used by educators to drive real-time, formative assessment in their classrooms. The software is used primarily by mathematics teachers to deliver homework, classwork and exams to their students. We have developed a computer adaptive test called PLACEments as an extension of ASSISTments to allow teachers to perform individual student assessment and by extension school-wide benchmarking. PLACEments uses a form of graph-based knowledge representation by which the exam results identify the specific mathematics skills that each student lacks. The system additionally provides differentiated practice determined by the students’ performance on the adaptive test. In this project, we describe the design and implementation of PLACEments as a skill assessment method and evaluate it in comparison with a fixed-item benchmark.