{"id":{"repo_id":"mtu","oai_identifier":"oai:digitalcommons.mtu.edu:etdr-3255"},"canonical_url":"https://search.ndltd.org/etd/mtu/oai:digitalcommons.mtu.edu:etdr-3255","repository":{"repo_id":"mtu","name":"Michigan Technological University","base_url":"https://digitalcommons.mtu.edu/do/oai/"},"display":{"title":"Beyond Satisfaction: Developing a Data Storytelling Evaluation Protocol For STEM Equity Program Assessment","abstract":"Higher education institutions routinely evaluate equity initiatives using measures of participation, engagement, and satisfaction. While these approaches provide useful information about program delivery, they offer limited insight into whether interventions change how participants understand institutional inequities. As a result, important dimensions of organizational learning and awareness often remain difficult to detect, particularly within STEM higher education environments where evaluation systems frequently emphasize measurable outputs over deeper indicators of change. This dissertation examines how awareness-related outcomes can be identified, measured, and communicated within equity intervention contexts. Using a three-study design, the research first presents a systematic review of equity intervention evaluation practices in STEM higher education, revealing a persistent reliance on satisfaction-based assessment approaches. It then develops and validates a perception-shift evaluation protocol using data from 248 participants across six iterations of Michigan Technological University's Advocates & Allies program. Findings demonstrate that participants consistently reported less favorable, yet more informed, perceptions of campus equity following workshop participation, suggesting that increased awareness may serve as an observable precursor to institutional change. Finally, the dissertation introduces the Data Storytelling Evaluation Protocol (DSEP), a computational framework designed to translate complex evaluation findings into formats accessible to institutional stakeholders. Collectively, the findings challenge the assumption that participant satisfaction is an adequate indicator of intervention effectiveness and demonstrate the value of measuring awareness as an evaluative outcome. The dissertation contributes conceptual, methodological, and computational approaches that support more meaningful evaluation of equity initiatives and provide institutions with new ways to recognize, interpret, and respond to evidence of change.","abstract_html":"Higher education institutions routinely evaluate equity initiatives using measures of participation, engagement, and satisfaction. While these approaches provide useful information about program delivery, they offer limited insight into whether interventions change how participants understand institutional inequities. As a result, important dimensions of organizational learning and awareness often remain difficult to detect, particularly within STEM higher education environments where evaluation systems frequently emphasize measurable outputs over deeper indicators of change. This dissertation examines how awareness-related outcomes can be identified, measured, and communicated within equity intervention contexts. Using a three-study design, the research first presents a systematic review of equity intervention evaluation practices in STEM higher education, revealing a persistent reliance on satisfaction-based assessment approaches. It then develops and validates a perception-shift evaluation protocol using data from 248 participants across six iterations of Michigan Technological University&#x27;s Advocates &amp; Allies program. Findings demonstrate that participants consistently reported less favorable, yet more informed, perceptions of campus equity following workshop participation, suggesting that increased awareness may serve as an observable precursor to institutional change. Finally, the dissertation introduces the Data Storytelling Evaluation Protocol (DSEP), a computational framework designed to translate complex evaluation findings into formats accessible to institutional stakeholders. Collectively, the findings challenge the assumption that participant satisfaction is an adequate indicator of intervention effectiveness and demonstrate the value of measuring awareness as an evaluative outcome. The dissertation contributes conceptual, methodological, and computational approaches that support more meaningful evaluation of equity initiatives and provide institutions with new ways to recognize, interpret, and respond to evidence of change.","abstract_has_math":false,"creators":["Colbert, Karen"],"institution":null,"degree_name":"Doctor of Philosophy in Computational Science and Engineering (PhD)","degree_level":"Open Access Dissertation","degree_discipline":"Department of Social Sciences","degree_department":null,"school":null,"contributors":["Mark Rouleau"],"advisors":[],"committee_chairs":[],"committee_members":["Michelle Jarvie-Eggart","Adrienne Minerick","Laura Brown"],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:19:32Z","subjects":["Higher Education","Faculty Development","STEM equity","Professional Development","Education"],"languages":[],"rights":[],"rights_urls":["http://creativecommons.org/licenses/by-nc/4.0/"],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.mtu.edu/etdr/2186"],"render_values":[{"text":"https://digitalcommons.mtu.edu/etdr/2186","href":"https://digitalcommons.mtu.edu/etdr/2186","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.37099/mtu.dc.etdr/2186","outbound_label":"DOI","outbound_source":"dc:identifier","record_url":"https://doi.org/10.37099/mtu.dc.etdr/2186","record_label":"Repository record","record_kind":"doi","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.mtu.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.mtu.edu%3Aetdr-3255","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Mark Rouleau"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Michelle Jarvie-Eggart","Adrienne Minerick","Laura Brown"]},{"key":"dc:creator","label":"Author","values":["Colbert, Karen"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.created","label":"Dc Date Created","values":["2026-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Department of Social Sciences"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Open Access Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy in Computational Science and Engineering (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Higher Education","Faculty Development","STEM equity","Professional Development","Education"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights.license","label":"Dc Rights License","values":["http://creativecommons.org/licenses/by-nc/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["10.37099/mtu.dc.etdr/2186","https://digitalcommons.mtu.edu/etdr/2186"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Higher education institutions routinely evaluate equity initiatives using measures of participation, engagement, and satisfaction. While these approaches provide useful information about program delivery, they offer limited insight into whether interventions change how participants understand institutional inequities. As a result, important dimensions of organizational learning and awareness often remain difficult to detect, particularly within STEM higher education environments where evaluation systems frequently emphasize measurable outputs over deeper indicators of change. This dissertation examines how awareness-related outcomes can be identified, measured, and communicated within equity intervention contexts. Using a three-study design, the research first presents a systematic review of equity intervention evaluation practices in STEM higher education, revealing a persistent reliance on satisfaction-based assessment approaches. It then develops and validates a perception-shift evaluation protocol using data from 248 participants across six iterations of Michigan Technological University's Advocates & Allies program. Findings demonstrate that participants consistently reported less favorable, yet more informed, perceptions of campus equity following workshop participation, suggesting that increased awareness may serve as an observable precursor to institutional change. Finally, the dissertation introduces the Data Storytelling Evaluation Protocol (DSEP), a computational framework designed to translate complex evaluation findings into formats accessible to institutional stakeholders. Collectively, the findings challenge the assumption that participant satisfaction is an adequate indicator of intervention effectiveness and demonstrate the value of measuring awareness as an evaluative outcome. The dissertation contributes conceptual, methodological, and computational approaches that support more meaningful evaluation of equity initiatives and provide institutions with new ways to recognize, interpret, and respond to evidence of change."]},{"key":"dc:title","label":"Title","values":["Beyond Satisfaction: Developing a Data Storytelling Evaluation Protocol For STEM Equity Program Assessment"]}]}],"canonical_facts":{"dc:contributor":["Mark Rouleau"],"dc:contributor.committeemember":["Michelle Jarvie-Eggart","Adrienne Minerick","Laura Brown"],"dc:creator":["Colbert, Karen"],"dc:date.created":["2026-01-01T08:00:00Z"],"dc:description.abstract":["Higher education institutions routinely evaluate equity initiatives using measures of participation, engagement, and satisfaction. While these approaches provide useful information about program delivery, they offer limited insight into whether interventions change how participants understand institutional inequities. As a result, important dimensions of organizational learning and awareness often remain difficult to detect, particularly within STEM higher education environments where evaluation systems frequently emphasize measurable outputs over deeper indicators of change. This dissertation examines how awareness-related outcomes can be identified, measured, and communicated within equity intervention contexts. Using a three-study design, the research first presents a systematic review of equity intervention evaluation practices in STEM higher education, revealing a persistent reliance on satisfaction-based assessment approaches. It then develops and validates a perception-shift evaluation protocol using data from 248 participants across six iterations of Michigan Technological University's Advocates & Allies program. Findings demonstrate that participants consistently reported less favorable, yet more informed, perceptions of campus equity following workshop participation, suggesting that increased awareness may serve as an observable precursor to institutional change. Finally, the dissertation introduces the Data Storytelling Evaluation Protocol (DSEP), a computational framework designed to translate complex evaluation findings into formats accessible to institutional stakeholders. Collectively, the findings challenge the assumption that participant satisfaction is an adequate indicator of intervention effectiveness and demonstrate the value of measuring awareness as an evaluative outcome. The dissertation contributes conceptual, methodological, and computational approaches that support more meaningful evaluation of equity initiatives and provide institutions with new ways to recognize, interpret, and respond to evidence of change."],"dc:identifier":["10.37099/mtu.dc.etdr/2186","https://digitalcommons.mtu.edu/etdr/2186"],"dc:rights.license":["http://creativecommons.org/licenses/by-nc/4.0/"],"dc:subject":["Higher Education","Faculty Development","STEM equity","Professional Development","Education"],"dc:title":["Beyond Satisfaction: Developing a Data Storytelling Evaluation Protocol For STEM Equity Program Assessment"],"thesis:degree_discipline":["Department of Social Sciences"],"thesis:degree_level":["Open Access Dissertation"],"thesis:degree_name":["Doctor of Philosophy in Computational Science and Engineering (PhD)"]},"updated_at":"2026-11-19T08:19:32Z"}