Predicting student performance is an important topic in academia, especially so in programming context, where identification of struggling students allows teachers to offer early and continuous assistance to help them improve their performance. It is thus essential to analyze student programming behavior to detect those at-risk students. This thesis uses data generated from 220 students in a master’s level programming course at a large European university. The students run unit tests in order to test their code when solving assignments, with a snapshot being taken of each test as it is executed. Unit testing is a method of testing software where individual units of source code are tested for correctness. A data set with simple features is derived from a database of snapshots and labeled with students’ grades. Then, the machine learning models support vector machine (SVM), naive Bayes (NB), random forest, and neural networks with one, two and three hidden layers each are trained, evaluated and performance is compared. The results show that SVM and neural networks models are likely the best performing all-rounders, with a possible naive Bayes selection depending on what goal one has. The thesis contributes by training machine learning models on students’ programming behavior. By arming teacher with models such as these, more students that need assistance can get in-time support and thus improve their performance. Future work can improve the models by using or combining other types of student data as features or use a larger data set. / Att förutsäga studenters prestationer är ett viktigt ämne inom akademin, särskilt i programmeringssammanhang, där identifiering av studenter som kämpar med sina studier gör det möjligt för lärare att erbjuda tidig och kontinuerlig hjälp för att hjälpa dem att förbättra sina prestationer. Det är därför viktigt att analysera studenternas programmeringsbeteende för att upptäcka dessa studenter som är vid risk. Denna uppsats använder data från 220 studenter i en programmeringskurs på masternivå vid ett stort europeiskt universitet. Studenterna kör enhetstester för att testa sin kod när de löser uppgifter, och en snapshot tas av varje test när det körs. Enhetstestning är en metod för att testa programvara där enskilda enheter av källkoden testas för korrekthet. En datamängd med enkla features härleds från en databas med snapshots och märks med studenternas betyg. Därefter tränas och utvärderas maskininlärningsmodellerna support vector machine (SVM), naive Bayes (NB), random forest och neurala nätverk med ett, två och tre dolda lager vardera och deras prestanda jämförs. Resultaten visar att SVM och neurala nätverk sannolikt är de bäst presterande allroundmodellerna, med ett möjligt naivt Bayes-val beroende på vilket mål man har. Uppsatsen bidrar genom att träna maskininlärningsmodeller på studenters programmeringsbeteende. Genom att utrusta lärare med modeller som dessa kan fler studenter som behöver hjälp få stöd i tid och därmed förbättra sina prestationer. Framtida arbete kan förbättra modellerna genom att använda eller kombinera andra typer av studentdata som features eller använda en större datamängd.

Two of the major goals in Educational Data Mining are determining students’ state of knowledge and determining their affective state. It is useful to be able to determine whether a student is engaged with a tutor or task in order to adapt to his/her needs and necessary to have an idea of the students' knowledge state in order to provide material that is appropriately challenging. These two problems are usually examined separately and multiple methods have been proposed to solve each of them. However, little work has been done on examining both of these states in parallel and the combined effect on a student’s performance. The work reported in this thesis explores ways to observe both behavior and performance in order to more fully understand student state.

Over the last two decades, the field of educational data mining (EDM) has been focusing on predicting the correctness of the next student response to the question (e.g., [2, 6] and the 2010 KDD Cup), in other words, predicting student short-term performance. Student modeling has been widely used for making such inferences. Although performing well on the immediate next problem is an indicator of mastery, it is by far not the only criteria. For example, the Pittsburgh Science of Learning Center's theoretic framework focuses on robust learning (e.g., [7, 10]), which includes the ability to transfer knowledge to new contexts, preparation for future learning of related skills, and retention - the ability of students to remember the knowledge they learned over a long time period. Especially for a cumulative subject such as mathematics, robust learning, particularly retention, is more important than short-term indicators of mastery. The Automatic Reassessment and Relearning System (ARRS) is a platform we developed and deployed on September 1st, 2012, which is mainly used by middle-school math teachers and their students. This system can help students better retain knowledge through automatically assigning tests to students, giving students opportunity to relearn the skill when necessary and generating reports to teachers. After we deployed and tested the system for about seven months, we have collected 287,424 data points from 6,292 students. We have created several models that predict students' retention performance using a variety of features, and discovered which were important for predicting correctness on a delayed test. We found that the strongest predictor of retention was a student's initial speed of mastering the content. The most striking finding was that students who struggled to master the content (took over 8 practice attempts) showed very poor retention, only 55% correct, after just one week. Our results will help us advance our understanding of learning and potentially improve ITS.

Les travaux de recherche de cette thèse concernent principalement l‘aide à la navigation individualisée dans un hypermédia épistémique. Nous disposons d‘un certain nombre de ressources qui peut se formaliser à l‘aide d‘un graphe acyclique orienté (DAG) : le graphe des épistèmes. Après avoir cerné les environnements de ressources et de parcours, les modalités de visualisation et de navigation, de traçage, d‘adaptation et de fouille de données, nous avons présenté une approche consistant à corréler les activités de conception ou d‘édition à celles dédiées à l‘utilisation et la navigation dans les ressources. Cette approche a pour objectif de fournir des mécanismes d‘individualisation de la navigation dans un environnement qui se veut évolutif. Nous avons alors construit des prototypes appropriés pour mettre à l‘épreuve le graphe des épistèmes. L‘un de ces prototypes a été intégré à une plateforme existante. Cet hypermédia épistémique baptisé HiPPY propose des ressources et des parcours portant sur l‘apprentissage du langage Python. Il s‘appuie sur un graphe des épistèmes, une navigation dynamique et un bilan de connaissances personnalisé. Ce prototype a fait l‘objet d‘une expérimentation qui nous a donné la possibilité d‘évaluer les principes introduits et d‘analyser certains usages. / This research work mainly concerns means of assistance in individualized navigation through an epistemic hypermedia. We have a number of resources that can be formalized by a directed acyclic graph (DAG) called the graph of epistemes. After identifying resources and pathways environments, methods of visualization and navigation, tracking, adaptation and data mining, we presented an approach correlating activities of design or editing with those dedicated to resources‘ use and navigation. This provides ways of navigation‘s individualization in an environment which aims to be evolutive. Then, we built prototypes to test the graph of epistemes. One of these prototypes was integrated into an existing platform. This epistemic hypermedia called HiPPY provides resources and pathways on Python language. It is based on a graph of epistemes, a dynamic navigation and a personalized knowledge diagnosis. This prototype, which was experimented, gave us the opportunity to evaluate the introduced principles and analyze certain uses.

abstract: Computer supported collaborative learning (CSCL) has made great inroads in classroom teaching marked by the use of tools and technologies to support and enhance collaborative learning. Computer mediated learning environments produce large amounts of data, capturing student interactions, which can be used to analyze students’ learning behaviors (Martinez-Maldonado et al., 2013a). The analysis of the process of collaboration is an active area of research in CSCL. Contributing towards this area, Meier et al. (2007) defined nine dimensions and gave a rating scheme to assess the quality of collaboration. This thesis aims to extract and examine frequent patterns of students’ interactions that characterize strong and weak groups across the above dimensions. To achieve this, an exploratory data mining technique, differential sequence mining, was employed using data from a collaborative concept mapping activity where collaboration amongst students was facilitated by an interactive tabletop. The results associate frequent patterns of collaborative concept mapping process with some of the dimensions assessing the quality of collaboration. The analysis of associating these patterns with the dimensions of collaboration is theoretically grounded, considering aspects of collaborative learning, concept mapping, communication, group cognition and information processing. The results are preliminary but still demonstrate the potential of associating frequent patterns of interactions with strong and weak groups across specific dimensions of collaboration, which is relevant for students, teachers, and researchers to monitor the process of collaborative learning. The frequent patterns for strong groups reflected conformance to the process of conversation for dimensions related to “communication” aspect of collaboration. In terms of the concept mapping sub-processes the frequent patterns for strong groups reflect the presentation phase of conversation with processes like talking, sharing individual maps while constructing the groups concept map followed by short utterances which represents the acceptance phase. For “joint information processing” aspect of collaboration, the frequent patterns for strong groups were marked by learners’ contributing more upon each other’s work. In terms of the concept mapping sub-processes the frequent patterns were marked by learners adding links to each other’s concepts or working with each other’s concepts, while revising the group concept map. / Dissertation/Thesis / Masters Thesis Computer Science 2015

abstract: EMBRACE (Enhanced Moved By Reading to Accelerate Comprehension in English) is an IPad application that uses the Moved By Reading strategy to help improve the reading comprehension skills of bilingual (Spanish speaking) English Language Learners (ELLs). In EMBRACE, students read the text of a story and then move images corresponding to the text that they read. According to the embodied cognition theory, this grounds reading comprehension in physical experiences and thus is more engaging. In this thesis, I used the log data from 20 students in grades 2-5 to design a skill model for a student using EMBRACE. A skill model is the set of knowledge components that a student needs to master in order to comprehend the text in EMBRACE. A good skill model will improve understanding of the mistakes students make and thus aid in the design of useful feedback for the student.. In this context, the skill model consists of vocabulary and syntax associated with the steps that students performed. I mapped each step in EMBRACE to one or more skills (vocabulary and syntax) from the model. After every step, the skill level is updated in the model. Thus, if a student answered the previous step incorrectly, the corresponding skills are decremented and if the student answered the previous question correctly, the corresponding skills are incremented, through the Bayesian Knowledge Tracing algorithm. I then correlated the students’ predicted scores (computed from their skill levels) to their posttest scores. I evaluated the students’ predicted scores (computed from their skill levels) by comparing them to their posttest scores. The two sets of scores were not highly correlated, but the results gave insights into potential improvements that could be made to the system with respect to user interaction, posttest scores and modeling algorithm. / Dissertation/Thesis / Masters Thesis Computer Science 2016

El texto completo de este trabajo no está disponible en el Repositorio Académico UPC por restricciones de la casa editorial donde ha sido publicado. / This research proposes a prediction model that might help reducing the dropout rate of university students in Peru. For this, a three-phase predictive analysis model was designed which was combined with the stages proposed by the IBM SPSS Modeler methodology. Bayesian network techniques was compared with decision trees for their level of accuracy over other algorithms in an Educational Data Mining (EDM) scenario. Data were collected from 500 undergraduate students from a private university in Lima. The results indicate that Bayesian networks behave better than decision trees based on metrics of precision, accuracy, specificity, and error rate. Particularly, the accuracy of Bayesian networks reaches 67.10% while the accuracy for decision trees is 61.92% in the training sample for iteration with 8:2 rate. On the other hand, the variables athletic person (0.30%), own house (0.21%), and high school grades (0.13%) are the ones that contribute most to the prediction model for both Bayesian networks and decision trees.

The overall goal of this work is to support student success in computer science. First, I introduce BitFit, an ungraded practice programming tool built to provide students with a pressure-free environment to practice and build confidence working through weekly course material. BitFit was used in an introductory programming course (CSC 110) at the University of Victoria for 5 semesters in 2015 and 2016. The contributions of this work are a number of studies done analyzing the log data collected by BitFit over those years. First, I explore whether patterns can be identified in log data to differentiate successful from unsuccessful students, with a specific focus on identifying students at-risk of failure within the first few weeks of the semester. Next, I separate out only those students who struggle early in the semester, and examine their changes in programming behaviour over time. The goal behind the second study is to differentiate between transient and sustained struggling, in an attempt better understand the reasons successful students are able to overcome early struggles. Finally, I combine survey data with log data to explore whether students understand whether their study habits are likely to lead to success. Overall, this work provides insight into the factors contributing to behavioural change in an introductory programming course. I hope this information can aid educators in providing supportive intervention aimed at guiding struggling students towards more productive learning strategies. / Graduate / 0984 / 0525 / 0710 / aestey@uvic.ca

Submitted by William Justo Figueiro (williamjf) on 2015-07-28T20:32:18Z No. of bitstreams: 1 27d.pdf: 4180188 bytes, checksum: 617cda1d8cedaa910bb66676e96c33d1 (MD5) / Made available in DSpace on 2015-07-28T20:32:18Z (GMT). No. of bitstreams: 1 27d.pdf: 4180188 bytes, checksum: 617cda1d8cedaa910bb66676e96c33d1 (MD5) Previous issue date: 2014-04-15 / Nenhuma / Aplicações que fazem uso de tecnologias como Mineração de Dados Educacionais (MDE) e Learning Analytics (LA) vêm sendo adotadas na mitigação da evasão escolar, disponibilizando informações sobre os alunos que são utilizadas em intervenções pedagógicas. Os trabalhos estudados sobre a implementação destas aplicações priorizam a descrição das técnicas empregadas e existem poucas avaliações da sua utilização em larga escala, além da falta de detalhamento sobre as causas da evasão. Este trabalho apresenta um estudo de fatores envolvidos no fenô- meno de evasão escolar e descreve a utilização de um sistema para MDE e LA durante 18 meses em cursos de graduação na modalidade de Educação a Distância. É ampliada a análise dos fatores tradicionalmente monitorados e utilizados nos sistemas de MDA e LA, com a inclusão de elementos associados ao papel exercido pelos docentes e pelo conjunto de aspectos metodológicos de cada instituição. O sistema possui como diferencial a flexibilidade na integração e utilização dos dados gerados no processo de mediação digital, o que permite que necessidades de diferentes ferramentas de apoio sejam disponibilizadas. Resultados positivos destacados são a identificação de perfis de alunos evasores e a realização de intervenções pedagógicas, com redução das médias da evasão. / Educational Data mining (EDM) and Learning Analytics (LA) applications have been adopted in mitigation of dropout, providing information about students who are employed in pedagogical interventions. The most papers about the implementation of these systems describe the techniques employed, there are few evaluations of their large-scale use, apart from the lack of detail about the causes of dropout. This work presents a study of factors involved in dropout and describes the use of a system for EDM and LA during 18 months for undergraduate courses in distance education. The analysis of the factors traditionally monitored and used in EDM and LA systems is extended, with the inclusion of elements associated with the role exercised by the teachers and by institutional methodological aspects. The system has flexibility in integration and use of data generated in the process of digital mediation, which allows different support tools to be available. Some results are the identification of evaders students profiles and the realization of pedagogical actions with reducing evasion.

abstract: Due to large data resources generated by online educational applications, Educational Data Mining (EDM) has improved learning effects in different ways: Students Visualization, Recommendations for students, Students Modeling, Grouping Students, etc. A lot of programming assignments have the features like automating submissions, examining the test cases to verify the correctness, but limited studies compared different statistical techniques with latest frameworks, and interpreted models in a unified approach. In this thesis, several data mining algorithms have been applied to analyze students’ code assignment submission data from a real classroom study. The goal of this work is to explore and predict students’ performances. Multiple machine learning models and the model accuracy were evaluated based on the Shapley Additive Explanation. The Cross-Validation shows the Gradient Boosting Decision Tree has the best precision 85.93% with average 82.90%. Features like Component grade, Due Date, Submission Times have higher impact than others. Baseline model received lower precision due to lack of non-linear fitting. / Dissertation/Thesis / Masters Thesis Computer Science 2019