"Intelligent tutoring systems rely on student modeling to understand student behavior. The result of student modeling can provide assessment for student knowledge, estimation of student¡¯s current affective states (ie boredom, confusion, concentration, frustration, etc), prediction of student performance, and suggestion of the next tutoring steps. There are three focuses of this dissertation. The first focus is on better predicting student performance by adding more information, such as student identity and information about how many assistance students needed. The second focus is to analyze different performance and feature set for modeling student short-term knowledge and longer-term knowledge. The third focus is on improving the affect detectors by adding more features. In this dissertation I make contributions to the field of data mining as well as educational research. I demonstrate novel Bayesian networks for student modeling, and also compared them with each other. This work contributes to educational research by broadening the task of analyzing student knowledge to student knowledge retention, which is a much more important and interesting question for researchers to look at. Additionally, I showed a set of new useful features as well as how to effectively use these features in real models. For instance, in Chapter 5, I showed that the feature of the number of different days a students has worked on a skill is a more predictive feature for knowledge retention. These features themselves are not a contribution to data mining so much as they are to education research more broadly, which can used by other educational researchers or tutoring systems. "

abstract: Reading comprehension is a critical aspect of life in America, but many English language learners struggle with this skill. Enhanced Moved by Reading to Accelerate Comprehension in English (EMBRACE) is a tablet-based interactive learning environment is designed to improve reading comprehension. During use of EMBRACE, all interactions with the system are logged, including correct and incorrect behaviors and help requests. These interactions could potentially be used to predict the child’s reading comprehension, providing an online measure of understanding. In addition, time-related features have been used for predicting learning by educational data mining models in mathematics and science, and may be relevant in this context. This project investigated the predictive value of data mining models based on user actions for reading comprehension, with and without timing information. Contradictory results of the investigation were obtained. The KNN and SVM models indicated that elapsed time is an important feature, but the linear regression models indicated that elapsed time is not an important feature. Finally, a new statistical test was performed on the KNN algorithm which indicated that the feature selection process may have caused overfitting, where features were chosen due coincidental alignment with the participants’ performance. These results provide important insights which will aid in the development of a reading comprehension predictor that improves the EMBRACE system’s ability to better serve ELLs. / Dissertation/Thesis / Masters Thesis Computer Science 2017

Com a popularização do uso de recursos tecnológicos na educação, uma enorme quantidade de dados, relacionados às interações entre alunos e esses recursos, é armazenada. Analisar esses dados, visando caracterizar os alunos, é tarefa muito importante, uma vez que os resultados dessa análise podem auxiliar professores no processo de ensino e aprendizagem. Entretanto, devido ao fato de as ferramentas utilizadas para essa caracterização serem complexas e pouco intuitivas, os profissionais da área de ensino acabam por não utilizá-las, inviabilizando a implementação de tais ferramentas em ambientes educacionais. Dentro desse contexto, a dissertação de mestrado aqui apresentada teve como objetivo analisar os dados provenientes de um sistema tutor inteligente, o MathTutor, que disponibiliza exercícios específicos de matemática, para identificar padrões de comportamento dos alunos que interagiram com esse sistema durante um determinado período. Essa análise foi realizada por meio de um processo de Mineração de Dados Educacionais (EDM), utilizando a ferramenta DAMICORE, com o intuito de possibilitar que fossem geradas, de forma rápida e eficaz, informações úteis à caracterização dos alunos. Durante a realização dessa análise, seguiram-se algumas fases do processo de descobrimento de conhecimento em bases de dados, seleção, pré-processamento, mineração dos dados e avaliação e interpretação. Na fase de mineração de dados, foi utilizada a ferramenta DAMICORE, que encontrou padrões que foram estudados na fase de avaliação e interpretação. A partir dessa análise foram encontrados padrões comportamentais dos alunos, por exemplo, alunos do sexo masculino apresentam rendimento superior ou inferior ao de alunas do sexo feminino e quais alunos terão um bom ou mau rendimento nas etapas finais do processo de ensino. Como principal resultado temos que uma das hipóteses criadas, Alunos que obtiveram bom desempenho no pós-teste imediato apresentaram dois dos três seguintes comportamentos: poucas interações na intervenção, baixo tempo interagindo com o sistema na intervenção e poucos misconceptions no pré-teste, teve sua acurácia comprovada dentre os dados utilizados nessa pesquisa. Assim, por meio desta pesquisa concluiu-se que a utilização da DAMICORE em contexto educacional pode auxiliar o professor a inferir o desempenho dos seus alunos oferecendo a ele a oportunidade de realizar as intervenções pedagógicas que auxiliem alunos com possíveis dificuldades e apresente novos desafios para aqueles com facilidade no tema estudado / With the popularization of the use of technological resources in education, a huge amount of data, related to the interactions between students and these resources, is stored. Analyzing this data, due to characterize the students, is an important task, since the results of this analysis can help teachers on teaching and learning process. However, due to the fact that the tools used to this characterization are complex and non-intuitive, the educational professionals do not use it, invalidating the implementation of such tools at educational environments. Within this context, this master\'s dissertation aimed analyzing the prevenient data from an educational web system named MathTutor, which offers specific math exercises to identify behavioral patterns of students who interacted with this system during some period. This analysis was performed by a process known as Educational Data Mining, using the tool named DAMICORE, in order to enable quickly and effectively the construction of helpful information to the characterization of the students. During the course of this analysis, some phases of the process of knowledge discovery in databases were followed: \"selection\", \"preprocessing\", \"data mining\" and \"evaluation and interpretation\". In \"data mining\" phase, the tool DAMICORE was used to find behavioral patterns of students which were studied at the \"evaluation and interpretation\" phase. From this analysis, behavioral patterns of students were found, for example, male students have higher or lower yield against the female students and which students are going to have a good or bad yield on the final steps of the educational process. As the main result we have one of the made assumptions, \"Students who get good performance in the \"immediate posttest\" showed two of the following behaviors: few interactions in the \"intervention\", low time interacting with the system in the \"intervention\" and few misconceptions in \"pretest\"\", has proven its accuracy among the data used in this dissertation. Thus, through this research, it was concluded that the use of DAMICORE at educational context can help teacher to infer the performance of their students offering him the opportunity to perform the pedagogical interventions that help students who faces difficulties and show new challenges for those who have facilities in the subject studied.

In the field of Artificial Intelligence in Education, many contributions have been made toward estimating student proficiency in Intelligent Tutoring Systems (cf. Corbett & Anderson, 1995). Although the community is increasingly capable of estimating how much a student knows, this does not shed much light on when the knowledge was acquired. In recent research (Baker, Goldstein, & Heffernan, 2010), we created a model that attempts to answer that exact question. We call the model P(J), for the probability that a student just learned from the last problem they answered. We demonstrated an analysis of changes in P(J) that we call “spikiness", defined as the maximum value of P(J) for a student/knowledge component (KC) pair, divided by the average value of P(J) for that same student/KC pair. Spikiness is directly correlated with final student knowledge, meaning that spikes can be an early predictor of success. It has been shown that both over-practice and under-practice can be detrimental to student learning, so using this model can potentially help bias tutors toward ideal practice schedules. After demonstrating the validity of the P(J) model in both CMU's Cognitive Tutor and WPI's ASSISTments Tutoring System, we conducted a pilot study to test the utility of our model. The experiment included a balanced pre/post-test and three conditions for proficiency assessment tested across 6 knowledge components. In the first condition, students are considered to have mastered a KC after correctly answering 3 questions in a row. The second condition uses Bayesian Knowledge Tracing and accepts a student as proficient once they earn a current knowledge probability (Ln) of 0.95 or higher. Finally, we test P(J), which accepts mastery if a student's P(J) value spikes from one problem and the next first response is correct. In this work, we will discuss the details of deriving P(J), our experiment and its results, as well as potential ways this model could be utilized to improve the effectiveness of cognitive mastery learning.

Intelligent Tutoring Systems (ITS) have been proven to be efficient in providing students assistance and assessing their performance when they do their homework. Many research projects have been done to analyze how students' knowledge grows and to predict their performance from within intelligent tutoring system. Most of them focus on using correctness of the previous question or the number of hints and attempts students need to predict their future performance, but ignore how they ask for hints and make attempts. In this research work, we build a Sequence of Actions (SOA) model taking advantage of the sequence of hints and attempts a student needed for previous question to predict students' performance. A two step modeling methodology is put forward in the work, which is a combination of Tabling method and the Logistic Regression. We used an ASSISTments dataset of 66 students answering a total of 34,973 problems generated from 5010 questions over the course of two years. The experimental results showed that the Sequence of Action model has reliable predictive accuracy than Knowledge Tracing and Assistance Model and its performance of prediction is improved after combining with Knowledge Tracing.

One of the main research challenges nowadays in the context of Massive Open Online Courses (MOOCs) is the automation of the evaluation process of text-based assessments effectively. Text-based assessments, such as essay writing, have been proved to be better indicators of higher level of understanding than machine-scored assessments (E.g. Multiple Choice Questions). Nonetheless, due to the rapid growth of MOOCs, text-based evaluation has become a difficult task for human markers, creating the need of automated systems for grading. In this thesis, we focus on the automated short answer grading task (ASAG), which automatically assesses natural language answers to open-ended questions into correct and incorrect classes. We propose an ensemble supervised machine learning approach that relies on two types of classifiers: a response-based classifier, which centers around feature extraction from available responses, and a reference-based classifier which considers the relationships between responses, model answers and questions. For each classifier, we explored a set of features based on words and entities. For the response-based classifier, we tested and compared 5 features: traditional n-gram models, entity URIs (Uniform Resource Identifier) and entity mentions both extracted using a semantic annotation API, entity mention embeddings based on GloVe and entity URI embeddings extracted from Wikipedia. For the reference-based classifier, we explored fourteen features: cosine similarity between sentence embeddings from student answers and model answers, number of overlapping elements (words, entity URI, entity mention) between student answers and model answers or question text, Jaccard similarity coefficient between student answers and model answers or question text (based on words, entity URI or entity mentions) and a sentence embedding representation. We evaluated our classifiers on three datasets, two of which belong to the SemEval ASAG competition (Dzikovska et al., 2013). Our results show that, in general, reference-based features perform much better than response-based features in terms of accuracy and macro average f1-score. Within the reference-based approach, we observe that the use of S6 embedding representation, which considers question text, student and model answer, generated the best performing models. Nonetheless, their combination with other similarity features helped build more accurate classifiers. As for response-based classifiers, models based on traditional n-gram features remained the best models. Finally, we combined our best reference-based and response-based classifiers using an ensemble learning model. Our ensemble classifiers combining both approaches achieved the best results for one of the evaluation datasets, but underperformed on the remaining two. We also compared the best two classifiers with some of the main state-of-the-art results on the SemEval competition. Our final embedded meta-classifier outperformed the top-ranking result on the SemEval Beetle dataset and our top classifier on SemEval SciEntBank, trained on reference-based features, obtained the 2nd position. In conclusion, the reference-based approach, powered mainly by sentence level embeddings and other similarity features, proved to generate the most efficient models in two out of three datasets and the ensemble model was the best on the SemEval Beetle dataset.

Submitted by Marta Toyoda (1144061@mackenzie.br) on 2018-02-09T19:36:53Z No. of bitstreams: 2 RENÊ DE ÁVILA MENDES.pdf: 2131022 bytes, checksum: 371eff9a643c4104cbd7ced2b556bab5 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Approved for entry into archive by Paola Damato (repositorio@mackenzie.br) on 2018-02-22T13:28:09Z (GMT) No. of bitstreams: 2 RENÊ DE ÁVILA MENDES.pdf: 2131022 bytes, checksum: 371eff9a643c4104cbd7ced2b556bab5 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Made available in DSpace on 2018-02-22T13:28:09Z (GMT). No. of bitstreams: 2 RENÊ DE ÁVILA MENDES.pdf: 2131022 bytes, checksum: 371eff9a643c4104cbd7ced2b556bab5 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Previous issue date: 2017-02-09 / To properly deal with volume, velocity and variety data dimensions in educational contexts is a major concern for Educational Institutions and both Educational Data Mining and Learning Analytics Researchers have cooperated to properly address this challenge which is popularly called Big Data. Hardware developments have been made to increase computing power, storage capacity and efficiency in energy use. New technologies in databases, file systems and distributed systems, as well as developments in data transmission techniques, data management, data analysis and visualization have been trying to overcome the challenge of processing, storing and analyzing large volumes of data and the inability to meet simultaneously the requirements of consistency, availability and tolerance of partitions. Although the architecture definition is the main task in a Big Data system design, objective guidelines for the selection of the architecture and the tools for the implementation of Big Data systems were not found in the literature. The present research aims to analyze the main architectures for both batch and stream processing and to use one of them in the construction of a Big Data environment, providing important orientations to Researchers, Technicians and Managers. Academic data and logs of the Virtual Learning Environment Moodle of an Academic Unit of a Higher Education Institution are used. / Lidar adequadamente com as dimensões de volume, velocidade e variedade dos dados no contexto educacional é um importante desafio para as Instituições de Ensino, e Pesquisadores das áreas de Mineração de Dados Educacionais e Learning Analytics têm cooperado para tratar adequadamente este desafio, popularmente chamado de Big Data. Desenvolvimentos em hardware têm sido feitos para aumentar o poder computacional, a capacidade de armazenamento e a eficiência no uso de energia. Novas tecnologias de bancos de dados, sistemas de arquivos e sistemas distribuídos, além do desenvolvimento de técnicas de transmissão, administração, análise e visualização de dados têm tentado vencer o desafio de processar, armazenar e analisar grandes volumes de dados e a impossibilidade de atender simultaneamente os requisitos de consistência, disponibilidade e tolerância a partições. Embora a definição da arquitetura seja a principal tarefa em um projeto de sistema Big Data, não foram encontradas na literatura orientações objetivas para a seleção da arquitetura e das ferramentas para a implementação de aplicações Big Data. A presente pesquisa tem por objetivo analisar as principais arquiteturas para processamento em lote e em fluxo e utilizar uma delas na construção de um ambiente Big Data, fornecendo importantes orientações a Pesquisadores, Técnicos e Gestores. São utilizados dados acadêmicos e logs do Ambiente Virtual de Aprendizagem Moodle de uma Unidade Acadêmica de uma Instituição de Ensino Superior.

Ingeniero Civil Matemático / En los últimos años, se ha abierto una oportunidad de hacer análisis más precisos de las habilidades y desempeños de los estudiantes. De a poco, han comenzado a proliferar sistemas de ejercitación en línea y tutores inteligentes que permiten registrar una gran cantidad de información valiosa referente al aprendizaje de los alumnos. La Minería de Datos Educacionales (MDE), es un campo de estudio dedicado a desarrollar métodos matemáticos para analizar datos provenientes de ambientes relacionados a la educación, y extraer la mayor cantidad de información para tratar de entender mejor a los estudiantes, profesores y actores relacionados, con el fin de mejorar los procesos educativos. En esta memoria se aborda el problema de predecir el desempeño de un alumno dados sus datos históricos recopilados a partir de su interacción en un sistema computacional de ejercitación en línea. Este desafío se ha constituido últimamente como uno de los más importantes dentro de la MDE, tal como evidencia el aumento de publicaciones relacionadas, y el gran interés que ha despertado por parte de universidades y entidades gubernamentales. En este trabajo, se analizan los registros almacenados de más de medio millón de ejercicios en línea realizados semanalmente en el 2011 por 805 estudiantes en 23 cursos de cuarto básico de 13 escuelas vulnerables, explorando varios de los enfoques más usados para enfrentar este problema, y proponiendo nuevas variantes para mejorar los resultados y ayudar a la detección de observaciones anómalas que podrían incluir instancias de "gaming the system". Adicionalmente, se estudia el problema de conocer cómo ciertos contenidos impactan en otros. Se trata de un problema de Minería de Datos Educacionales central en el diseño curricular y la planificación de clases. Usualmente esta red de influencias causales se construye en base a las opiniones de expertos. Algunos contribuyen explicitando la dependencia lógica de los contenidos y otros con sus experiencias personales al enseñar esos contenidos. Sin embargo, es muy importante contrastar esas opiniones con el proceso de aprendizaje que efectivamente ocurre en el aula y construir redes causales en base a la evidencia empírica. Aprovechamos los datos y técnicas de Minería de Datos para generar automáticamente la primera red causal de contenidos de un currículo construida empíricamente. Finalmente, se reporta el análisis del impacto de la ejercitación en línea en el desempeño de la prueba SIMCE. Mediciones en condiciones de laboratorio muestran que la ejercitación aumenta el aprendizaje. Sin embargo, implementaciones escolares no han mostrado impactos positivos. Este trabajo muestra la experiencia con escuelas vulnerables donde los estudiantes hacen decenas de ejercicios matemáticos semanales en un sistema en línea. El SIMCE de matemáticas subió significativamente, más de tres veces el aumento histórico logrado a nivel nacional en 2011. Además, los cursos que realizaron mayor cantidad de ejercicios lograron un mayor aumento en el SIMCE, independiente del efecto del profesor y de la escuela.

Com a popularização do uso de recursos tecnológicos na educação, uma enorme quantidade de dados, relacionados às interações entre alunos e esses recursos, é armazenada. Analisar esses dados, visando caracterizar os alunos, é tarefa muito importante, uma vez que os resultados dessa análise podem auxiliar professores no processo de ensino e aprendizagem. Entretanto, devido ao fato de as ferramentas utilizadas para essa caracterização serem complexas e pouco intuitivas, os profissionais da área de ensino acabam por não utilizá-las, inviabilizando a implementação de tais ferramentas em ambientes educacionais. Dentro desse contexto, a dissertação de mestrado aqui apresentada teve como objetivo analisar os dados provenientes de um sistema tutor inteligente, o MathTutor, que disponibiliza exercícios específicos de matemática, para identificar padrões de comportamento dos alunos que interagiram com esse sistema durante um determinado período. Essa análise foi realizada por meio de um processo de Mineração de Dados Educacionais (EDM), utilizando a ferramenta DAMICORE, com o intuito de possibilitar que fossem geradas, de forma rápida e eficaz, informações úteis à caracterização dos alunos. Durante a realização dessa análise, seguiram-se algumas fases do processo de descobrimento de conhecimento em bases de dados, seleção, pré-processamento, mineração dos dados e avaliação e interpretação. Na fase de mineração de dados, foi utilizada a ferramenta DAMICORE, que encontrou padrões que foram estudados na fase de avaliação e interpretação. A partir dessa análise foram encontrados padrões comportamentais dos alunos, por exemplo, alunos do sexo masculino apresentam rendimento superior ou inferior ao de alunas do sexo feminino e quais alunos terão um bom ou mau rendimento nas etapas finais do processo de ensino. Como principal resultado temos que uma das hipóteses criadas, Alunos que obtiveram bom desempenho no pós-teste imediato apresentaram dois dos três seguintes comportamentos: poucas interações na intervenção, baixo tempo interagindo com o sistema na intervenção e poucos misconceptions no pré-teste, teve sua acurácia comprovada dentre os dados utilizados nessa pesquisa. Assim, por meio desta pesquisa concluiu-se que a utilização da DAMICORE em contexto educacional pode auxiliar o professor a inferir o desempenho dos seus alunos oferecendo a ele a oportunidade de realizar as intervenções pedagógicas que auxiliem alunos com possíveis dificuldades e apresente novos desafios para aqueles com facilidade no tema estudado / With the popularization of the use of technological resources in education, a huge amount of data, related to the interactions between students and these resources, is stored. Analyzing this data, due to characterize the students, is an important task, since the results of this analysis can help teachers on teaching and learning process. However, due to the fact that the tools used to this characterization are complex and non-intuitive, the educational professionals do not use it, invalidating the implementation of such tools at educational environments. Within this context, this master\'s dissertation aimed analyzing the prevenient data from an educational web system named MathTutor, which offers specific math exercises to identify behavioral patterns of students who interacted with this system during some period. This analysis was performed by a process known as Educational Data Mining, using the tool named DAMICORE, in order to enable quickly and effectively the construction of helpful information to the characterization of the students. During the course of this analysis, some phases of the process of knowledge discovery in databases were followed: \"selection\", \"preprocessing\", \"data mining\" and \"evaluation and interpretation\". In \"data mining\" phase, the tool DAMICORE was used to find behavioral patterns of students which were studied at the \"evaluation and interpretation\" phase. From this analysis, behavioral patterns of students were found, for example, male students have higher or lower yield against the female students and which students are going to have a good or bad yield on the final steps of the educational process. As the main result we have one of the made assumptions, \"Students who get good performance in the \"immediate posttest\" showed two of the following behaviors: few interactions in the \"intervention\", low time interacting with the system in the \"intervention\" and few misconceptions in \"pretest\"\", has proven its accuracy among the data used in this dissertation. Thus, through this research, it was concluded that the use of DAMICORE at educational context can help teacher to infer the performance of their students offering him the opportunity to perform the pedagogical interventions that help students who faces difficulties and show new challenges for those who have facilities in the subject studied.

Studien fokuserar på att hitta olika mönster i närvarostatistik hos elever som inte närvarar i skolan. Informationen som resultatet ger kan därefter användas som ett beslutsunderlag för skolor eller till andra organisationer som är intresserade av EDM inom närvarostatistik. Arbetet genomförde en kvalitativ metodansats med en fallstudie som bestod utav en litteraturstudie samt en implementation. Litteraturstudien användes för att få en förståelse över vanliga tillvägagångssätt inom EDM, som därefter låg till grund för implementationen som använde arbetssättet CRISP-DM. Resultatet blev fem olika mönster som definieras genom dataanalys. Mönstren visar frånvaro ur ett tidsperspektiv samt per ämne och kan ligga till grund för framtida beslutsunderlag. / The study focuses on finding different patterns in attendance statistics for students who are not present at school. The information provided by the results can thereafter be used as a basis for decision-making for schools or for other organizations interested in EDM within attendance statistics. The work carried out a qualitative method approach with a case study that consisted a literature study and an implementation. The literature study was used to gain an understanding of common approaches within EDM, which subsequently formed the basis for the implementation that used the working method CRISP-DM. The project resulted in five different patterns defined by data analysis. The patterns show absence from a time perspective and per subject and can form the basis for future decision-making.