京都大学 / 新制・課程博士 / 博士(情報学) / 甲第24735号 / 情博第823号 / 新制||情||138(附属図書館) / 京都大学大学院情報学研究科社会情報学専攻 / (主査)教授 緒方 広明, 教授 伊藤 孝行, 教授 吉川 正俊 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DGAM

Det svenska skolsystemet genomgår en digitaliseringsprocess. I och med digitaliseringen har ett starkt intresse av att utforska olika typer av lärodata via olika läroplattformar utvecklats. För att analysera sådan data kan man ta hjälp av Learning Analytics (LA). LA är när man samlar, analyserar och rapporterar lärodata från diverse lärandeaktiviteter. För att analysera datan mer effektivt kan man ta hjälp av visualisering. Denna studie presenterar ett visualiseringsverktyg riktat till lärare. Det har utvecklats för att stödja lärare i deras förståelse av hur valda självreglerat lärande (SRL)strategier har utvärderats av studenter. Att kunna applicera SRL-strategier är av stor vikt eftersom studenter som lär sig att applicera dessa kommer att ha fördelar båda vid framtida studier och framtida arbete. Syftet med denna studie var att se hur lärodata kan visualiseras för att förstå studenters användning av lärostrategier och hur bra visualiseringsverktyget var ur ett användbarhetsperspektiv. Användbarheten av verktyget testades genom ett experiment där ett antal frågor ställdes. Svaren gick att finna i visualiseringen. Deltagarna i studien fick sedan svara på ett System Usabilityformulär. Resultaten visar att det utvecklade verktyget är användbart och att olika användare använder sig av, om möjligt, olika metoder för att komma fram till ett svar. Denna uppsats presenterar riktlinjer att ha i beaktande när ett visualiseringsverktyg som ska stödja lärare i att förstå studenters användning av SRL-strategier ska designas. / The Swedish school system is going through a digitalization process. With digitalization, a strong interest in exploring learner data available through various digital platforms has emerged. In order to analyse the data, one can take help of learning analytics (LA) which relates to collect, report and analyse data from learners or learning activities. To be able to analyse it in a more efficient way, one can take help from visualization. This study presents a visualisation tool aimed at teachers. It has in particular been developed to support teachers in their understanding of how chosen self-regulated learning (SRL) strategies were evaluated by students. SRL strategies are important to be able to apply since students who can learn to apply SRL strategies have an advantage during both future studies and work compared to those who cannot. The aim with this study was to see how learner data can be visualized in order to understand students' use of learning strategies and how good the developed visualization tool is from a usability perspective. The perceived usability of this tool was tested by means of interviews where a series of questions was asked where the answers could be found in the visualization tool. The participants also answered a System Usability Questionnaire. The results showed that the tool developed is usable and that different users of the tool tend to, if possible, use different methods to reach the answer. This thesis presents design guidelines to have in consideration when designing a visualization tool in order to aid teachers in analysing students' use of SRL strategies.

<p>  </p> <p>Clustering, a prevalent class of machine learning (ML) algorithms used in data mining and pattern-finding—has increasingly helped engineering education researchers and educators see and understand assessment patterns at scale. However, a challenge remains to make ML-enabled educational inferences that are useful and reliable for research or instruction, especially if those inferences influence pedagogical decisions or student outcomes. ML offers an opportunity to better personalizing learners’ experiences using those inferences, even within large engineering classrooms. However, neglecting to verify the trustworthiness of ML-derived inferences can have wide-ranging negative impacts on the lives of learners. </p> <p><br></p> <p>This study investigated what student clusters exist within the standard operational data of a large first-year engineering course (>1500 students). This course focuses on computational thinking skills for engineering design. The clustering data set included approximately 500,000 assessment data points using a consistent five-scale criterion-based grading framework. Two clustering techniques—N-TARP profiling and K-means clustering—examined criterion-based assessment data and identified student cluster sets. N-TARP profiling is an expansion of the N-TARP binary clustering method. N-TARP is well suited to this course’s assessment data because of the large and potentially high-dimensional nature of the data set. K-means clustering is one of the oldest and most widely used clustering methods in educational research, making it a good candidate for comparison. After finding clusters, their interpretability and trustworthiness were determined. The following research questions provided the structure for this study: RQ1 – What student clusters do N-TARP profiling and K-means clustering identify when applied to structured assessment data from a large engineering course? RQ2 – What are the characteristics of an average student in each cluster? and How well does the average student in each cluster represent the students of that cluster? And RQ3 – What are the strengths and limitations of using N-TARP and K-means clustering techniques with large, highly structured engineering course assessment data?</p> <p><br></p> <p>Although both K-means clustering and N-TARP profiling did identify potential student clusters, the clusters of neither method were verifiable or replicable. Such dubious results suggest that a better interpretation is that all student performance data from this course exist in a single homogeneous cluster. This study further demonstrated the utility and precision of N-TARP’s warning that the clustering results within this educational data set were not trustworthy (by using its W value). Providing this warning is rare among the thousands of available clustering methods; most clustering methods (including K-means) will return clusters regardless. When a clustering algorithm identifies false clusters that lack meaningful separation or differences, incorrect or harmful educational inferences can result. </p>

[EN] Access to the Spanish public university system is a complex process that involved administration, offering a certain number of places in different degrees that offer universities, and students, who must prioritize their preferences in a list. Determine what are the reasons that motivate a student, and their family and personal environment, to choose a particular degree and University to develop his studies, is a complex problem and that we have approached from the point of view of structural analysis. The aim of this work is to propose a partial multivariate model that can give an account of the weight of the different variables and identified factors involved in the decision about the choice of studies and University. The model we propose is specified based on a series of structural relationships involving a set of variables and context data, which we have grouped in the following factors: - Individual factors, associated with aspects related to student and personal interests, some academic and others clearly not. - Social factors, related with aspects that have to do with the social perception of the degree, their employability, the perception that we have of the University prestige, the degree, or the profession that gives access. - Context data, relative to the value of the access mark, the offer of seats or demand that has occurred in a certain degree and University in the years immediately prior to the taking of the decision. For this purpose a structural model and a questionnaire are proposed to evaluate the model, Model validation concludes with an extensive survey and analysis of the model results. / [ES] El acceso al Sistema Universitario Público Español es un proceso complejo en el que interviene la administración, ofreciendo un número determinado de plazas en las distintas titulaciones que ofertan las Universidades, y el estudiante, que debe priorizar en una lista sus preferencias. Determinar cuáles son las razones que mueven a un estudiante, y a su entorno familiar y personal, a elegir una determinada titulación y/o Universidad para cursar sus estudios superiores, es un problema complejo que hemos abordado desde el punto de vista del análisis estructural. El objetivo de este trabajo es proponer un modelo multivariado y parcial que pueda dar cuenta del peso de las distintas variables y factores identificados que influyen en la decisión acerca de la elección de estudios y universidad. El modelo propuesto se especifica en base a una serie de relaciones estructurales en las que intervienen un conjunto de variables y datos de contexto que hemos agrupado en los siguientes factores: - Factores individuales, vinculados a aspectos relacionados con el estudiante y sus intereses personales, algunos de carácter académico y otros claramente no. - Factores sociales, relacionados con aspectos que tienen que ver con la percepción social de la titulación, su empleabilidad, la percepción que se tiene del prestigio de la Universidad, del título, o de la profesión a la que da acceso.. - Datos de contexto, relativos al valor de la nota de corte de una titulación, a la oferta de plazas o a la demanda que se ha producido en una determinada titulación y universidad en los años inmediatamente anteriores al de la toma de la decisión. Para ello se propone un modelo estructural y se propone un cuestionario para evaluar las variables del modelo. La validación de modelo y cuestionario concluye con una amplia encuesta y el análisis de los resultados del modelo. / [CA] L'accés al Sistema Universitari Públic Espanyol és un procés complex en el qual intervé l'administració, oferint un nombre determinat de places en les diferents titulacions que ofereixen les universitats, i l'estudiant, que ha de prioritzar en una llista seves preferències. Determinar quines són les raons que mouen a un estudiant, i al seu entorn familiar i personal, a triar una determinada titulació i/o Universitat per cursar els seus estudis superiors, és un problema complex que hem abordat des del punt de vista de l'anàlisi estructural. L'objectiu d'aquest treball és proposar un model multivariat i parcial que puga donar compte del pes de les diferents variables i factors identificats que influeixen en la decisió sobre l'elecció d'estudis i universitat. El model proposat s'especifica en la base d'una sèrie de relacions estructurals en què intervenen un conjunt de variables i dades de context que hem agrupat en els següents factors: - Factors individuals, vinculats a aspectes relacionats amb l'estudiant i els seus interessos personals, alguns de caràcter acadèmic i altres clarament no. - Factors socials, relacionats amb aspectes que tenen a veure amb la percepció social de la titulació, la seva ocupabilitat, la percepció que es té del prestigi de la Universitat, del títol, o de la professió a la qual dóna accés .. - Dades de context, relatius al valor de la nota de tall d'una titulació, a l'oferta de places o la demanda que s'ha produït en una determinada titulació i universitat en els anys immediatament anteriors al de la presa de la decisió. Per a això es proposa un model estructural, es proposa un qüestionari per avaluar les variables del model. La validació del model conclou amb una àmplia enquesta i l'anàlisi dels resultats del model. / Soriano Jiménez, PP. (2016). Un modelo estructural para el análisis de los factores asociados a la elección de estudios universitarios [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/64076

Η διδακτορική διατριβή εντάσσεται στο πεδίο της Υπολογιστικά Υποστηριζόμενης Συνεργατικής Μάθησης, ΥΥΣΜ (Computer Supported Collaborative Learning, CSCL). ‘Eχει ως στόχο την ανάπτυξη και πρόταση μίας μεθόδου για την αυτοματοποιημένη ανάλυση, ταξινόμηση και αξιολόγηση της ποιότητας συνεργατικών εκπαιδευτικών δραστηριοτήτων. Αφενός βασίζεται σε ευρήματα ποιοτικής έρευνας και αφ’ ετέρου συνδυάζει την χρήση τεχνικών μηχανικής μάθησης και ανώτερων μαθηματικών που χρησιμοποιούνται ευρέως σε πλήθος άλλων ερευνητικών πεδίων, μελετώντας τους τρόπους που μπορούν να υιοθετηθούν και να συνεισφέρουν στο πεδίο της Υπολογιστικά Υποστηριζόμενης Συνεργατικής μάθησης (χρονοσειρές). Βασικό μέλημα είναι η προτεινόμενη μέθοδος να επιτρέπει την εξαγωγή χρήσιμων συμπερασμάτων για την ποιότητα των συνεργατικών δραστηριοτήτων με τρόπο ποσοτικό και αυτόματο ώστε να είναι δυνατή η χρήση της σε μεγάλα σύνολα δεδομένων. Η παρούσα μελέτη έδειξε πως η ποιότητα της συνεργασίας αποτυπώνεται στον τρόπο που κατανέμεται η συνεργατική δραστηριότητα στον χρόνο και η χρήση χρονοσειρών αποτυπώνει τα ιδιαίτερα χαρακτηριστικά της συνεργασίας με ικανοποιητικό τρόπο. Η μέθοδος αξιολογήθηκε ξεχωριστά αλλά και σε αντιπαράθεση με αντίστοιχα μοντέλα και μεθόδους. Η προτεινόμενη μέθοδος πλεονεκτεί ως προς την απλότητα κατασκευής και λειτουργίας ενώ διαπιστώθηκε με στατιστικά σημαντικό τρόπο η εγκυρότητα των αποτελεσμάτων της. H μέθοδος δεν απαιτεί την ολοκλήρωση της δραστηριότητας αλλά ενδείκνυται και για την αξιολόγησή της σε πραγματικό χρόνο. Οι χρονοσειρές δραστηριότητας περιγράφουν ικανοποιητικά βασικές συνεργατικές διαστάσεις που ορίζονται ως «χαμηλού επιπέδου» όπως η Επικοινωνία και η Κοινή Επεξεργασία Πληροφορίας και οι οποίες θεωρείται ότι αντικατοπτρίζονται ιδιαίτερα στην διαλογική δραστηριότητα που εκτυλίσσεται μεταξύ των συνεργατών. Από την άλλη, για διαστάσεις ανωτέρου επιπέδου που αντιπροσωπεύονται από πιο περίπλοκες δομές αλληλεπίδρασης, όπως για παράδειγμα ο Συντονισμός και η Διαπροσωπική Σχέση μεταξύ συνεργαζόμενων μερών, οι χρονοσειρές δραστηριότητας δεν καταφέρνουν να τις αποτυπώσουν ικανοποιητικά. Αποδείχθηκε στατιστικά πως για την συγκεκριμένη περίπτωση συνεργατικών μαθησιακών δραστηριοτήτων οι ουσιαστικές αλληλεπιδράσεις μεταξύ των χρηστών που υποδηλώνουν μεταφορά και οικοδόμηση κοινής γνώσης και κοινού τόπου επικοινωνίας αλλά και σχέσεις αιτιότητας, είναι δυνατόν να ανιχνευθούν μέσα σε χρονικά παράθυρα μικρού μεγέθους, της τάξης των τριάντα δευτερολέπτων. Σύνθετες δομές που αποτυπώνουν την δημιουργία στρατηγικής προσέγγισης ή διαμοιρασμού χώρου και χρόνου και απαιτούν μεγαλύτερους χρόνους εξέλιξης, δεν είναι δυνατόν να αποτυπωθούν επαρκώς με αυτή την προσέγγιση. Η έρευνα αυτή δεν αποσκοπεί κατά κανένα τρόπο στην αντικατάσταση της ανθρώπινης κρίσης ή γενικότερα του ανθρώπινου παράγοντα αλλά αντίθετα επιδιώκει να υποστηρίξει το έργο του. / The PhD thesis is part of ongoing research in the field of Computer-Supported Collaborative Learning (CSCL). The main contribution of this thesis is to design and propose a method for the automatic analysis, classification and evaluation of the quality of collaboration of learning activities. On one hand, the method is based and reflects the findings of qualitative research and on the other hand, it uses machine learning algorithms and statistical methods that allow the quantitative analysis of data. We used modeling techniques widely used in other scientific fields (time series) and studied how they can be used in CSCL to contribute new knowledge. The objective of the study is to implement a method for the representation, classification and evaluation of collaborative activities. It was shown that the quality of collaboration and its fundamental aspects is portrayed in the way the activity itself is distributed in time. It was shown through visualizations and statistical analysis that time series allow the effective representation of collaboration and its qualitative characteristics. The classification and evaluation method that was proposed is supported by a machine-learning model. The model was further evaluated as an automated rater of collaboration quality and compared to other similar models. The advantage of the proposed method over others is the simple structure and low-cost, as well as the potential to be used in real-time. The proposed approach attempts to describe and portray the interaction of users through their concurrent activity on different but common workspaces. For that reason we make use of common, basic activity metrics and time series. The time series of activity can describe successfully low level construct such as Communication and Information Processing. For more advanced and complicated constructs however, such as Coordination and Interpersonal Relationship, time series could not capture adequately the qualitative characteristics and underlying mechanisms. This finding comes in agreement with similar studies that point out the need of combined analysis methods that will use in combination content analysis techniques and natural language processing. It was also shown that in the particular context, the meaningful interactions that point to constructive collaboration, successful knowledge building and reciprocal activity can be mapped in small time frames, of about 30 seconds. More complicated structures that signify e.g. strategy planning and effective coordination, take more time to unfold and therefore cannot be traced in such small time frames. This study does not attempt in any way to substitute or overcome the human judgment and human factor, either in the analysis or teaching activity. On the contrary, we believe that the teacher cannot be replaced by automated tools and methods but should be supported and empowered.

Nowadays, interactive visualizations have been one of the most used tools in Big Data fields for the purpose of searching for relationships and structured information in large datasets of unstructured information. In this project, these tools are applied to extract structured information from students following Self-Regulated Learning (SRL). By means of an interactive graph, we are able to study the paths that the students follow in the learning materials. Our visualization supports the investigation of patterns of behaviour of the students, which later could be used, for example, to adapt the study program to the student’s needs in a dynamic way or offer guidance if necessary.

Purpose – This paper is part of an extensive action research project on learning analytics and focuses on the analysis criteria in Virtual Collaborative Learning (VCL) settings. We analyse how the efficiency of virtual learning facilitation can be increased by (semi-) automated learning analytics. Monitoring items are the starting point that enable the learning facilitator to identify learning problems and deduce adequate actions of intervention. However, the sophisticated media-based learning environment does not allow monitoring of vast amounts of items and appreciate the learning processes simultaneously. Design/methodology/approach – This paper fulfils the sub-goal of selecting and prioritising monitoring items for e-collaboration. The procedure is split into two Research Questions (RQ). A specification of the monitoring items will be compiled by a comparison and a consolidation of the already existing monitoring sheets. Therefore, we interviewed the responsible docents on differences and similarities. Additionally, we coded each monitoring item inductively due to their monitoring objective. As a result, we reduced the monitoring sheets to 40 final monitoring items (RQ1). In order to prioritise them, the learning facilitators scored the relevance and the complexity of the collection and assessment of data using a questionnaire. The analysis focused on differences in understanding of relevance and complexity. Further, we identified the highest scored monitoring items as well as scores with leverage potential. Afterwards we prioritised the items based on the applied analysis (RQ2). Originality/value – While previous studies on learning analytics were mostly driven by the educational data mining field and as a consequence had a technological focus. This paper is based on an existing pedagogical concept of VCL and therefore prioritises monitoring items to be implemented as selected learning analytics. Hence, it is guaranteed that the analysis is related directly to the learning content. Practical implications – This research paper achieved two outcomes: Firstly, a courseindependent standardised monitoring sheet. Thus, the reduction of the monitoring items should simplify and objectify the observation and clarify the performance review. Secondly, an insight into the relevance of each monitoring item had been delivered to the facilitators and provides significance on the quality of e-collaboration. Furthermore, the complexity score shows the necessary effort for data collection and assessment while the combination of relevance and complexity scores leads to the prioritisation of the needs of (semi-) automated learning analytics to support the learning facilitation.

In the last decade, development of Information and Communication Technology (ICT) thatsupports online learning has increased the demand for e-learning and Massive Open OnlineCourses (MOOCs). Despite their increased popularity, MOOCs are struggling with highdropout rates and only a small percentage of learners complete the courses they enrolled in. Thepurpose of this thesis is to gain knowledge about MOOC learner behaviour. The aim of thestudy is to identify the motivations of learners and how these differ between learners whocompleted a course and those who dropped out. Research on MOOC learners has mostly beencarried out using a quantitative approach. While quantitative methodologies are effective inhandling the large amount of data produced by MOOCs, qualitative methods can give deeperinsights into online learners’ motivations. Therefore, this thesis employs an explanatorysequential mixed methods research, in which sentiment analysis and topic modeling of learnerreviews from the platform Coursera are further explained by qualitative interviews with MOOClearners. In the study 28,000 reviews scraped from five courses within the fields of data sciencewere analyzed and ten interviews were held with learners who either completed, dropped outfrom or both completed and dropped out from a MOOC. In the quantitative analysis nine coursefactors were found that learners wrote about: content, delivery, assessment, learning experience,tools, video material, teaching style, instructor skills and course provider. In addition, eighteenthemes were yielded from the interviews: self-discipline, just for fun, certificates, personaldevelopment, knowledge, career, time, equipment, practical exercise, interaction, instructor,reality, structure, external material, cost, community, degree of difficulty and other. In thediscussion the empirical findings are reflected upon using the theoretical framework of theresearch and the literature review. The result does not reveal any differences in motivationsbetween learners who completed a course and those who dropped out, however, it does identifyfactors that caused learners’ to drop out and the topics that most negative learner reviews wereabout. This research contributes to the body of knowledge in the field of research on MOOClearner retention and motivations. The topic is relevant for research in education informaticsand for continued improvements in delivery of MOOCs.

Since the advent of the Internet, we have been observing a fast-paced development within the computing world. One of the major innovations in recent years is the “Internet of Things”, which brings interconnectedness between devices and humans to unprecedented heights. This technological breakthrough enabled the emergence of a new sub-field within Learning Analytics, Multimodal Learning Analytics, which makes use of several types of data sources to study learning-related processes. As computers and sensors become increasingly cheaper and more accessible,  research within this new sub-field grows, yet some gaps remain unexplored. Additionally, there is a research bias toward computer-assisted learning environments, rather than physical ones. At the same time, the current labor market is highly competitive, and possessing profession-related skills is not sufficient to land a job. Besides these skills, there is an increasing demand for social skills, such as communication, teamwork, and collaboration. However, there is a gap between the skills that are trained in an academic setting and the ones that are required by the labor market. Having this background in mind, this work aims at designing and evaluating an IoT sensor system capable of tracking patterns observed under social interactions within a group, and more specifically, in terms of the distance between group members while solving a task. Another important aspect of this study is the system's cost-effectiveness so that it can be employed in a scalable and sustainable manner. To achieve this goal, a multimethodological approach for Design Science Research was adopted, which implied the combination of several methods such as sketching, prototyping, and testing. As a result, this study contributes both to the research area of Multimodal Learning Analytics, and to educational practices.

Learning Analytics plays an increasing role in the analysis of virtual learning activities. This article addresses the gap between educational needs and technical supply. By means of a Systematic Literature Review of the LAK conferences the authors extracted observations, methods and tools which represent potential solutions for a given eCollaboration scenario. Based on three prioritised examples of an observation sheet, methods are derived and recommendations for the use of Learning Analytics tools are given. The result is a catalogue that enable users to select suitable methods and tools for an implementation. The (semi-) automation can increase the efficiency of Community Managers in monitoring the participants and hence make real-time intervention feasible.