{"id":{"repo_id":"msu","oai_identifier":"oai:d.lib.msu.edu:etd_53243"},"canonical_url":"https://search.ndltd.org/etd/msu/oai:d.lib.msu.edu:etd_53243","repository":{"repo_id":"msu","name":"Michigan State University","base_url":"https://d.lib.msu.edu/oai"},"display":{"title":"Analyzing STEM Instructional Change Strategies with Data Science and Machine Learning Methods","abstract":"Thesis (Ph.D.)--Michigan State University. Computational Mathematics, Science and Engineering - Doctor of Philosophy, 2026","abstract_html":"Thesis (Ph.D.)--Michigan State University. Computational Mathematics, Science and Engineering - Doctor of Philosophy, 2026","abstract_has_math":false,"creators":["Bolger, Emily"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":["Caballero, Marcos D","Green, Jennifer","O'Shea, Brian","Yadav, Aman","Beach, Andrea"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2026,"date_issued":"2026","date_published":"2026","updated_at":"2026-11-18T18:59:44Z","subjects":["Computer science","Education"],"languages":["eng"],"rights":["Attribution-NonCommercial 4.0 International"],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["etd:53243","local:Bolger_grad.msu_0128D_22289"],"render_values":[{"text":"etd:53243","href":null,"code":true},{"text":"local:Bolger_grad.msu_0128D_22289","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://doi.org/doi:10.25335/7949-ag56","outbound_label":"DOI","outbound_source":"dc:identifier","record_url":"https://doi.org/doi:10.25335/7949-ag56","record_label":"Repository record","record_kind":"doi","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://d.lib.msu.edu/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Ad.lib.msu.edu%3Aetd_53243","prefix":"oai_dc"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Caballero, Marcos D","Green, Jennifer","O'Shea, Brian","Yadav, Aman","Beach, Andrea"]},{"key":"dc:creator","label":"Author","values":["Bolger, Emily"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2026"]},{"key":"dc:relation","label":"Dc Relation","values":["Electronic Theses & Dissertations"]},{"key":"dc:type","label":"Dc Type","values":["text","Electronic dissertations"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer science","Education"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Attribution-NonCommercial 4.0 International"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["etd:53243","local:Bolger_grad.msu_0128D_22289","https://doi.org/doi:10.25335/7949-ag56"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Thesis (Ph.D.)--Michigan State University. Computational Mathematics, Science and Engineering - Doctor of Philosophy, 2026","For over half a century, national policy and national funding streams have allowed education researchers to develop and assess changes to classroom instruction in science education. Until 2011, it was mostly unclear what curricular change strategies were being implemented and the impact these changes have on student learning. To assess the current state of the field, Henderson et al. (2011) conducted a systematic literature review of 191 articles published in undergraduate science education from 1995 to 2008. They constructed a four-square topology of instructional change strategies across the intended outcome and the aspect of the system to be changed. Their efforts challenged researchers to use evidence-based practices and shifted the focus of funding opportunities in the field (e.g., the NSF Improving Undergraduate STEM Education (IUSE)). An example of one such systemic change effort is the Partnership for Integration of Computing into Undergraduate Physics (PICUP), founded in 2015. The PICUP community assists educators with the barriers that arise from designing computing curricula for non-computing classrooms. These efforts, in turn, support student learning and success as it affords the opportunity to learn skills required for current STEM careers. The community used Slack as an asynchronous communication platform to discuss ideas and challenges outside of the formal professional development programs. Using Social Network Analysis (SNA), we explore the patterns of communication between the members of the PICUP community. We represent each channel in the Slack Workspace with a network, encoding members as nodes and interactions between members as weighted, directed edges. We use metrics like strength, weighted reciprocity, and closeness centrality, to evaluate whether the Slack community aligns with PICUP's community goals through our defined conceptual framework. We observe three levels of user engagement: active, passive, and receiver. We conclude the Slack communication system is not conducive to sustained community engagement as users do not frequently appear in multiple channels and tend to stay in the same user engagement level across channels. We make recommendations to the PICUP community including a reassessment of the intentions of the Slack Workspace in the community and an extension of the work to include a qualitative analysis of the Slack messages.PICUP is one of many novel approaches to systemic change initiatives in undergraduate STEM education. With the increased number of initiatives, it is imperative to conduct a follow-up systematic literature review (SLR) to Henderson et al. (2011) comparing the effects of change strategies and associated student success. However, the number of publications on instructional change strategies in STEM education makes it challenging to accurately and efficiently evaluate the field through a methodological SLR. We leverage a human-in-the-loop framework to analyze the integration of machine learning methods for thematic extraction in SLRs. Our human-in-the-loop implementation demonstrates the importance of interdisciplinary teams to leverage domain expertise when applying computational methods in disciplinary contexts. Through an iterative process, experts in the field can provide insights on the data which can be utilized to refine machine learning models. While this process requires additional time and training of non-machine learning team members, it is critical for aligning machine learning outputs with project goals. Further, to assess the current capabilities of thematic analysis models, we present a comparison of a traditional thematic analysis method, Latent Dirichlet Allocation (LDA), with pre-trained text embedding models with dimension reduction and unsupervised clustering through a Computational Grounded Theory framework. We find that pre-trained text embedding models more efficiently identify cohesive themes. While not all themes are applicable to project goals, they allow for validation of the human qualitative codes, identified independently through the SLR methodology. Through this dissertation, we use data science and machine learning methods to analyze systemic change efforts in the STEM education community - both in a localized and comprehensive scale. The work establishes guidelines and recommendations for integrating data science and machine learning methods into education research settings.","Description based on online resource. Title from PDF t.p. (Michigan State University Fedora Repository, viewed ).","Includes bibliographical references."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf","1 online resource (174 p.) : ill."]},{"key":"dc:title","label":"Title","values":["Analyzing STEM Instructional Change Strategies with Data Science and Machine Learning Methods"]}]}],"canonical_facts":{"dc:contributor":["Caballero, Marcos D","Green, Jennifer","O'Shea, Brian","Yadav, Aman","Beach, Andrea"],"dc:creator":["Bolger, Emily"],"dc:date":["2026"],"dc:description":["Thesis (Ph.D.)--Michigan State University. Computational Mathematics, Science and Engineering - Doctor of Philosophy, 2026","For over half a century, national policy and national funding streams have allowed education researchers to develop and assess changes to classroom instruction in science education. Until 2011, it was mostly unclear what curricular change strategies were being implemented and the impact these changes have on student learning. To assess the current state of the field, Henderson et al. (2011) conducted a systematic literature review of 191 articles published in undergraduate science education from 1995 to 2008. They constructed a four-square topology of instructional change strategies across the intended outcome and the aspect of the system to be changed. Their efforts challenged researchers to use evidence-based practices and shifted the focus of funding opportunities in the field (e.g., the NSF Improving Undergraduate STEM Education (IUSE)). An example of one such systemic change effort is the Partnership for Integration of Computing into Undergraduate Physics (PICUP), founded in 2015. The PICUP community assists educators with the barriers that arise from designing computing curricula for non-computing classrooms. These efforts, in turn, support student learning and success as it affords the opportunity to learn skills required for current STEM careers. The community used Slack as an asynchronous communication platform to discuss ideas and challenges outside of the formal professional development programs. Using Social Network Analysis (SNA), we explore the patterns of communication between the members of the PICUP community. We represent each channel in the Slack Workspace with a network, encoding members as nodes and interactions between members as weighted, directed edges. We use metrics like strength, weighted reciprocity, and closeness centrality, to evaluate whether the Slack community aligns with PICUP's community goals through our defined conceptual framework. We observe three levels of user engagement: active, passive, and receiver. We conclude the Slack communication system is not conducive to sustained community engagement as users do not frequently appear in multiple channels and tend to stay in the same user engagement level across channels. We make recommendations to the PICUP community including a reassessment of the intentions of the Slack Workspace in the community and an extension of the work to include a qualitative analysis of the Slack messages.PICUP is one of many novel approaches to systemic change initiatives in undergraduate STEM education. With the increased number of initiatives, it is imperative to conduct a follow-up systematic literature review (SLR) to Henderson et al. (2011) comparing the effects of change strategies and associated student success. However, the number of publications on instructional change strategies in STEM education makes it challenging to accurately and efficiently evaluate the field through a methodological SLR. We leverage a human-in-the-loop framework to analyze the integration of machine learning methods for thematic extraction in SLRs. Our human-in-the-loop implementation demonstrates the importance of interdisciplinary teams to leverage domain expertise when applying computational methods in disciplinary contexts. Through an iterative process, experts in the field can provide insights on the data which can be utilized to refine machine learning models. While this process requires additional time and training of non-machine learning team members, it is critical for aligning machine learning outputs with project goals. Further, to assess the current capabilities of thematic analysis models, we present a comparison of a traditional thematic analysis method, Latent Dirichlet Allocation (LDA), with pre-trained text embedding models with dimension reduction and unsupervised clustering through a Computational Grounded Theory framework. We find that pre-trained text embedding models more efficiently identify cohesive themes. While not all themes are applicable to project goals, they allow for validation of the human qualitative codes, identified independently through the SLR methodology. Through this dissertation, we use data science and machine learning methods to analyze systemic change efforts in the STEM education community - both in a localized and comprehensive scale. The work establishes guidelines and recommendations for integrating data science and machine learning methods into education research settings.","Description based on online resource. Title from PDF t.p. (Michigan State University Fedora Repository, viewed ).","Includes bibliographical references."],"dc:format":["application/pdf","1 online resource (174 p.) : ill."],"dc:identifier":["etd:53243","local:Bolger_grad.msu_0128D_22289","https://doi.org/doi:10.25335/7949-ag56"],"dc:language":["eng"],"dc:relation":["Electronic Theses & Dissertations"],"dc:rights":["Attribution-NonCommercial 4.0 International"],"dc:subject":["Computer science","Education"],"dc:title":["Analyzing STEM Instructional Change Strategies with Data Science and Machine Learning Methods"],"dc:type":["text","Electronic dissertations"]},"updated_at":"2026-11-18T18:59:44Z"}