京都大学 / 新制・課程博士 / 博士(情報学) / 甲第24934号 / 情博第845号 / 新制||情||141(附属図書館) / 京都大学大学院情報学研究科社会情報学専攻 / (主査)教授 緒方 広明, 教授 伊藤 孝行, 教授 田島 敬史 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

No description available.

Competency-based medical education (CBME) is a paradigm of assessing resident performance through well-defined tasks, objectives and milestones. A large number of data points are generated during a five-year period as a resident accomplishes the assigned tasks. However, no tool support exists to process this data for early identification of a resident-at-risk failing to achieve future milestones. In this thesis, the implementation of CBME at McMaster's Royal College Emergency Medicine residency program was studied and the development of a machine learning algorithm (MLA) to identify patterns in resident performance was reported. The adaptivity of multiple MLAs to build a tool support for monitoring residents' progress and flagging those who are in most need of assistance in the context of emergency medicine education was evaluated. / Thesis / Master of Science (MSc)

The purpose of this study is to analyze which behaviors are or are not helpful for debugging when a novice is in a state of unproductive persistence. Further, this project will exploratorily use a variety of analytical techniques -- including association rule mining, process mining, frequent sequence mining, and machine learning-- in order to determine which approaches are useful for data analysis. For the study, programming process data from hundreds of novice programmers were analyzed to determine which behaviors were more or less likely to be correlated with escaping a state of unproductive persistence. Of these events, only three had a statistically significant difference in their rates of occurrence and large effect sizes: file, edit, and compile events. While the data set cannot reveal a user's motivation for a file event, the most logical explanation of these events is that the user is tracing the code. Thus, a higher rate of file events suggests that code tracing (with the goal of code comprehension) is a key behavior correlated with a student's ability to escape a state of unproductive persistence. On the other hand, editing events are far more common in unproductive states that are not escaped. A content analysis suggests that there are more trivial edits for users in an unescaped state of unproductive persistence. An important finding of this study is that an unproductive persistence is not just a phenomenon of the worst-performing students; rather, a third of users who completed the assignment had at least one unproductive state. This study also lends support to the idea that tinkering combined with code tracing is correlated with positive outcomes, but that less systematic tinkering is not effective behavior. Further, association rule mining and frequent sequence mining were effective tools for data analysis in this study. The findings from this study have two main practical implications for curriculum designers and instructors: (1) the need to normalize struggle and (2) possibilities for curriculum and tool development. This work is particularly important given that debugging is not normally a process evident to instructors, curriculum designers, tool developers, and computer science education researchers, either because it happens outside of class time and/or because it is a process and these stakeholders usually only see the end result; this project attempts to make the process of debugging more transparent.

Student interactions in learning environments are vital for learning development. The growth of online learning in higher education has led stakeholders to question how to identify student interactions with course material and increase the quality and value of the learning experience. This research focused on leveraging existing learning analytics from the Canvas Learning Management System (LMS) to identify course interactions and make data-informed course design decisions. Learning analytics were collected from 113 students in three course sections of an online construction management course. Three surveys were also distributed to each course section to gather the students' perceptions of the learning methods and their interactions for assistance. An exploratory graphical analysis visually depicted student interactions in the online course through the students' hourly and weekly interaction levels, page visits, and discussion board activity. A paired t-test was used to statistically compare the survey responses on the students' perceptions of the learning methods. The learning analytics results showed the students' interaction levels peaked in the afternoon and evening hours, and their weekly interactions and page visits lessened after the midterm exam. Additionally, based on Pearson's correlation test, the discussion board interactions significantly correlated with student performance. Lastly, the surveys showed that students found watching the lecture videos and reading the lecture slides to be the most helpful methods when learning the course material. These results have important implications for online stakeholders as learning analytics and student perceptions can inform online course design to facilitate student, instructor, and content interactions. / Master of Science / In an online course, students click on lecture pages to watch lecture videos, they use discussion boards to post and reply to their peers, and they visit their courses at whatever time suits them. These interactions are difficult for an instructor to identify. Therefore, making it harder for them to engage with the students, determine which students are at-risk for failing, or develop their courses based on the students' interactions. This research study leverages learning analytics to identify student interactions in an online construction management course to improve academic decision-making and course design. Learning analytics are interaction data collected from a course that includes every student's interaction with the course material (e.g., page clicks, discussion posts and replies). Additionally, surveys were distributed to each of the three online construction management course sections used in this study to gather the students' thoughts about the available learning methods (e.g., video lectures, lecture slides). The learning analytics results showed that student interaction fluctuates by the hour and lessens after the midterm exam. The survey results found watching the lecture videos and reading the lecture slides were the most helpful learning methods. The capabilities of learning analytics must be addressed by online stakeholders when developing future online courses. The growth of online learning is inevitable, and the results of this paper suggest that learning analytics can identify unnoticed student interaction patterns and influence future online course design.

This dissertation illuminates patterns across disparate university data sets to identify the insights that may be gained through the analysis of large amounts of disconnected student data on first-year engineering (FYE) students and to understand how FYE instructors use data to inform their teaching practices. Grounded by the Academic Plan Model, which highlights student characteristics as an important consideration in curriculum development, the study brings together seemingly distinct pieces of information related to students' learning, engagement with class resources, and motivation so that faculty may better understand the characteristics and activities of students enrolled in their classes. In the dissertation's first manuscript, I analyzed learning management system (LMS) timestamp log-files from 876 students enrolled in the FYE course during Fall 2013. Following a series of quantitative analyses, I discovered that students who use the LMS more frequently are more likely to have higher grades within the course. This finding suggests that LMS usage might be a way to understand how students interact with course materials outside of traditional class time. Additionally, I found differential relationships between LMS usage and course performance across different instructors as well as a relationship between timing of LMS use and students' course performance. For the second manuscript, I connected three distinct data sets: FYE student's LMS data, student record data, and FYE program survey data that captured students' motivation and identity as engineers at two time points. Structural equation modeling results indicate that SAT Math was the largest predictor of success in the FYE course, and that students' beginning of semester engineering expectancy was the only significant survey construct to predict final course grade. Finally, for the third manuscript I conducted interviews with eight FYE instructors on how they use student data to inform their teaching practices. Ten themes emerged which describe the limited explicit use of formal data, but many instructors use data on an informal basis to understand their students. Findings also point to specific, existing data that the university already collects that could be provided to instructors on an aggregate, class-level basis to help them better understand their students. / Ph. D.

Online learning provides secondary students with the flexibility to meet learning goals at a time, place, and pace that meets their needs. In order to be successful in online courses, students must exhibit strong levels of self-regulated learning (SRL), including time management, goal setting, and resilience. Understanding the SRL behaviors of secondary online students can help course designers, instructors, and administrators design effective learning environments and provide targeted support to help students be successful in online learning environments. This three-manuscript dissertation analyzes the course pacing behaviors of secondary students enrolled in online courses to attain insight on their self-regulated behaviors. As a proxy for understanding online self-regulated behaviors, temporal variables were examined including the average length of an online study session, the amount of time between study sessions, the average number of study session sand the overall number of study sessions for each participant. The first study investigated the relationships between student demographics and temporal behaviors in relation to their effect on the depth of student understanding on the course midterm exam. The second study explores trends in online temporal behaviors, including the consistency and frequency of engagement in the course, and how these behaviors change over time in relation to student demographics. The final article utilizes student demographics and temporal behaviors to study their effect on academic achievement, as measured by their performance on the Advanced Placement exam. Findings from these studies indicate that online pacing behavior is related to student demographics and previous academic experience while also providing insight into how these variables affect achievement. By understanding the temporal behaviors of online secondary students, personalized support can be provided to strengthen student time management and engagement to promote academic achievement.

<p>Social presence positively influences the motivation, satisfaction, retention, and learning outcomes of online students. Although it is crucial for successful online learning experiences, little work has thoroughly examined the evolution of social presence over time and the influence of social presence on peer interaction. In other words, if social presence can be learned by interacting with others. This three-article dissertation study elucidates this gap by answering the overarching question: How does online students' social presence evolve over time to shape their online learning behaviors? Using stochastic-actor oriented models to reflect the dependence among learners in online collaborative learning communities, this dissertation investigated how learners' social presence evolved in learner-learner interaction resulting in two empirical studies and one conceptual framework. The first study explored social presence through clickstream interaction (e.g., number of replies received/sent in an online discussion) of 382 learners enrolled in a Massive Open Online Course. Three key findings from the study were: 1) dropout rates could be lowered if social presence affordances are used purposefully; 2) adding social media characteristics to online discussion boards, for example, "like" buttons, inhibits conversational behavior, and eventually, decreases achievement of learning outcomes; and 3) the "rich-get-richer" effect also applies to social presence, reinforcing highly active students' behavior and risking inactive online students to experience isolation. The second study used peer-nomination data (i.e., asking students who they interact with) and a scale to investigate the spread of social presence perceptions in online networks of students over three consecutive courses (n = 197). Although there was no evidence of social influence, online learners who nominated more peers are more likely to report higher social presence perceptions over time. Students were not more likely to share with those who showed similar levels of social presence. The "rich-get-richer effect" was observed in the incoming nominations of learners. The third study is a conceptual framework that integrates network theory and the online learning literature into a new perspective to analyze learners' online behaviors and interactions under the light of social presence theory. The proposed framework includes four main steps: 1) interaction, 2) social presence alignment, 3) unit of analysis definition, and 3) network statistics and inferential analysis selection. The findings of this dissertation improve educational practice by identifying behaviors that harm online social presence and providing specific actions for online instructors and instructional designers to promote social presence in online learning.  </p>

Recent research have been focusing on supporting teachers in the classroom. Such support has been shown to benefit from the development and employment of teacher-facing analytic dashboards to help them to make fast and effective decisions in regard to the in-class student learning activities. The evolving interest in this field has facilitated the emergence of the Teaching Analytics area of practice and research. However, current research efforts have indicated that the use of such dashboards usually adds another layer to the already dynamic and complex situation for teachers, which can divert their attention and can often be experienced as a disturbing factor in the class. Therefore, it is highly important to examine how such teacher-facing dashboards can be improved from the user experience perspective, in a way that would allow teachers to grasp student learning activities easily and with good perceived usability. The aim of this study is to understand how we can better design teacher-facing dashboards to more adequately support K-12 teachers in their decisions that would provide relevant in-time and student support. The study applies Nielsen's three-round iterative design approach to understand the existing usability problems and further develop the dashboard, originally designed by the company. In order to investigate users’ perceived attitude towards the redesigned dashboard, the final prototype has been evaluated through a Technology Acceptance Model questionnaire and semi-structured interviews with nine participants. As a result, the redesigned teacher-faced dashboard was proven to have a high potential to support teachers’ decisions. The efficiency of the Technology Acceptance Model was verified and put into general context on how tools for teachers should be designed for the usage in the classroom. Additionally, some major challenges for teachers with using external tools during class were discovered and are discussed in the context of a newly designed dashboard. / Befintlig forskning stödjer lärare i klassrummet genom att utveckla analytiska visualiseringsverktyg (a.k.a. dashboards) som lärare kan använda för att fatta snabba och effektiva beslut med avseende på elevernas läraktiviteter. Det växande intresset för detta område har lett till framväxten av Teaching Analytics-fältet inom praktik och forskning. Forskning har dock visat att användandet av dessa verktyg vanligtvis lägger till ytterligare ett lager till den redan dynamiska och komplexa situationen för lärare, vilket kan avleda deras uppmärksamhet och ofta fungera som en störande faktor i klassrummet. Därför är det mycket viktigt att undersöka hur sådana visualiseringsverktyg för lärare kan förbättras ur användarperspektiv, på ett sätt som skulle göra det möjligt för lärare att förstå elevernas läraktiviteter enkelt och med god upplevd användbarhet. Syftet med denna studie är att förbättra användargränssnittet för ett befintligt, så att det på ett mer adekvat sätt kan stödja lärare i sina beslut och erbjuda relevant stöd till eleverna. Studien tillämpar Nielsens tre-rundors iterativa designmetod för att förstå de befintliga användbarhetsproblemen och vidareutveckla en existerande dashboard, ursprungligen utvecklad av företaget. För att undersöka användarnas inställning till det omdesignade verktyget har den slutliga prototypen utvärderats genom ett frågeformulär och semistrukturerade intervjuer med nio deltagare. Resultat visar att det omdesignade de verktyget har en stor potential för att stödja lärarnas beslut i klassrummet. Effektiviteten för Teknik Acceptant Modellen (TAM) verifieras och sattes i allmän kontext för hur olika verktyg för lärare bör utformas för användning i klassrummet. Dessutom diskuteras lärarnas stora utmaningar med att använda externa verktyg under lektioner i samband med ny verktyget.

Submitted by William Justo Figueiro (williamjf) on 2015-07-28T20:32:18Z No. of bitstreams: 1 27d.pdf: 4180188 bytes, checksum: 617cda1d8cedaa910bb66676e96c33d1 (MD5) / Made available in DSpace on 2015-07-28T20:32:18Z (GMT). No. of bitstreams: 1 27d.pdf: 4180188 bytes, checksum: 617cda1d8cedaa910bb66676e96c33d1 (MD5) Previous issue date: 2014-04-15 / Nenhuma / Aplicações que fazem uso de tecnologias como Mineração de Dados Educacionais (MDE) e Learning Analytics (LA) vêm sendo adotadas na mitigação da evasão escolar, disponibilizando informações sobre os alunos que são utilizadas em intervenções pedagógicas. Os trabalhos estudados sobre a implementação destas aplicações priorizam a descrição das técnicas empregadas e existem poucas avaliações da sua utilização em larga escala, além da falta de detalhamento sobre as causas da evasão. Este trabalho apresenta um estudo de fatores envolvidos no fenô- meno de evasão escolar e descreve a utilização de um sistema para MDE e LA durante 18 meses em cursos de graduação na modalidade de Educação a Distância. É ampliada a análise dos fatores tradicionalmente monitorados e utilizados nos sistemas de MDA e LA, com a inclusão de elementos associados ao papel exercido pelos docentes e pelo conjunto de aspectos metodológicos de cada instituição. O sistema possui como diferencial a flexibilidade na integração e utilização dos dados gerados no processo de mediação digital, o que permite que necessidades de diferentes ferramentas de apoio sejam disponibilizadas. Resultados positivos destacados são a identificação de perfis de alunos evasores e a realização de intervenções pedagógicas, com redução das médias da evasão. / Educational Data mining (EDM) and Learning Analytics (LA) applications have been adopted in mitigation of dropout, providing information about students who are employed in pedagogical interventions. The most papers about the implementation of these systems describe the techniques employed, there are few evaluations of their large-scale use, apart from the lack of detail about the causes of dropout. This work presents a study of factors involved in dropout and describes the use of a system for EDM and LA during 18 months for undergraduate courses in distance education. The analysis of the factors traditionally monitored and used in EDM and LA systems is extended, with the inclusion of elements associated with the role exercised by the teachers and by institutional methodological aspects. The system has flexibility in integration and use of data generated in the process of digital mediation, which allows different support tools to be available. Some results are the identification of evaders students profiles and the realization of pedagogical actions with reducing evasion.