A novel decision-theoretic architecture for intelligent tutoring systems, DT Tutor (DT), was fleshed out into a complete ITS and evaluated. DT uses a dynamic decision network to probabilistically look ahead to anticipate how its tutorial actions will influence the student and other aspects of the tutorial state. It weighs its preferences regarding multiple competing objectives by the probabilities that they will occur and then selects the tutorial action with maximum expected utility.
The evaluation was conducted in two phases. First, logs were recorded from interactions of students with a Random Tutor (RT) that was identical to DT except that it selected randomly from relevant tutorial actions. The logs were used to learn many of DTs key probabilities for its model of the tutorial state. Second, the logs were replayed to record the actions that DT and a Fixed-Policy Tutor (FT) would select for a large sample of scenarios. FT was identical to DT except that it selected tutorial actions by emulating the fixed policies of Cognitive Tutors, which are theoretically based, widely used, and highly effective. The possible action selections for each scenario were rated by a panel of judges who were skilled human tutors. The main hypotheses tested were that DTs action selections would be rated higher than FTs and higher than RTs. This was the first comparison of a decision-theoretic tutor with a non-trivial competitor.
DT was rated higher than FT overall and for all subsets of scenarios except help requests, for which it was rated equally. DT was also rated much higher than RT. The judges preferred that the tutors provide proactive help and the study design permitted this information to be put to use right away to develop and evaluate enhanced versions of DT and FT. The enhanced versions of DT and FT were rated about equally and higher than non-enhanced DT except on help requests. The variability of the actions selected by both non-enhanced and enhanced versions of DT demonstrated more sensitivity to the tutorial state than the actions selected by non-enhanced and enhanced versions of FT.
| Identifer | oai:union.ndltd.org:PITT/oai:PITTETD:etd-08182005-131235 |
| Date | 05 October 2005 |
| Creators | Murray, Robert Charles |
| Contributors | Marek J. Druzdzel, Kurt A. VanLehn, Gregory F. Cooper, Kevin D. Ashley |
| Publisher | University of Pittsburgh |
| Source Sets | University of Pittsburgh |
| Language | English |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | http://etd.library.pitt.edu/ETD/available/etd-08182005-131235/ |
| Rights | unrestricted, I hereby certify that, if appropriate, I have obtained and attached hereto a written permission statement from the owner(s) of each third party copyrighted matter to be included in my thesis, dissertation, or project report, allowing distribution as specified below. I certify that the version I submitted is the same as that approved by my advisory committee. I hereby grant to University of Pittsburgh or its agents the non-exclusive license to archive and make accessible, under the conditions specified below, my thesis, dissertation, or project report in whole or in part in all forms of media, now or hereafter known. I retain all other ownership rights to the copyright of the thesis, dissertation or project report. I also retain the right to use in future works (such as articles or books) all or part of this thesis, dissertation, or project report. |
Page generated in 0.0017 seconds