Many universities across the United States have been experiencing an increased demand for computer science majors. Adjusting curriculum to meet increased demand runs the risk of damaging ongoing efforts to broaden participation in computer science. To manage growth, and increase the representation of women and underrepresented minorities in the field, we must first understand current patterns for participation, and factors that may impact access and persistence. Universities with common first-year engineering programs present an opportunity for addressing some of the barriers that have traditionally limited access to computer science to certain groups. In particular, common first-year programs could provide early positive experiences with computer programming which encourage more students to consider computer science as a viable major. To better understand how a common first-year engineering program may impact matriculation and persistence in computer science, I conducted studies to identify high-level patterns of participation in computer science, as well as how students experience programming instruction in an introductory engineering course. All studies share the same context: a large public research institution with a common first-year engineering program. Results indicate that women are leaving computer science at all points of the curriculum, contributing to a reduced representation of women earning CS degrees. In contrast, URM and first-generation students have higher representation at graduation than when declaring major interest before the start of their first year. / Doctor of Philosophy / Many universities across the United States have been experiencing an increased demand for computer science majors. Adjusting curriculum to meet demand runs the risk of damaging efforts to increase the diversity of the computer science workforce. To manage growth and increase the representation of women and underrepresented minorities (students who are not white or East Asian) in the field, we must first understand who currently studies computer science, and factors that lead to their success in the major. Universities with general first-year engineering programs present an opportunity for addressing some of the barriers that have traditionally discouraged women and underrepresented minorities from pursuing computer science. In particular, these programs could provide early positive experiences with computer programming which encourage more students to consider computer science as a possible major. To better understand how experiences during students’ first-year transition to college may impact decisions to major in computer science, I conducted studies to explore who enters computer science, and how they succeed in the major, as well as how students experience programming instruction in an introductory engineering course. All studies share the same context: a large public research institution with a general first-year engineering program. Results indicate that women are leaving computer science at all points of the curriculum, contributing to a reduced representation of women earning CS degrees. In contrast, underrepresented minority students and students with parents who did not receive a college degree, make up a higher percentage in the group graduating with a CS degree than in the group who declare CS as their first major.

Until recently, most educational tools have worked in silos. If a teacher wanted her students to complete small programming exercises, record videos, and collaborate through discussion boards, three disconnected tools were probably needed. Learning Tools Interoperability (LTI) is a communication protocol that enables different learning tools to talk to each other and share scores with a Learning Management System (LMS). While most commercial LMS now support LTI, most educational software developed by small research efforts do not. This is often because of the lack of resources needed to understand the working of LTI and the process of using LTI in their applications. Our aim is to encourage the use of LTI within the CS Education community. We have developed tutorials that include example applications. We also provide a use case of how LTI is implemented in the OpenDSA eTextbook system. As another use case, we have enabled auto-grading of Jupyter Notebook assignments by providing immediate feedback to students and updating scores to the Canvas gradebook. We provide a Jupyter plugin to upload notebook files to the Web-CAT auto-grading system. We integrate Aalto University's ACOS content into OpenDSA as a third use case. / Master of Science / Until recently, most educational tools have worked in silos. If a teacher wanted her students to complete small programming exercises, record videos, and collaborate through discussion boards, three disconnected tools were probably needed. These disconnected tools did not integrate with the Learning Management Systems (LMS), such as Canvas and Moodle. Instructors had to manually manage these separate tools and enter scores into the LMS. There are standards such as Learning Tools Interoperability (LTI) that these learning tools can implement to enable them to talk to each other and to share scores with an LMS. However, most educational software developed by small research efforts do not support LTI. This is often because of the lack of resources needed to understand the working of LTI and the process of using LTI in their applications. We aim to encourage the use of LTI within the CS Education community. We have developed tutorials that include example applications. We also provide a use case of how LTI is implemented in OpenDSA, an eTextbook system developed at Virginia Tech. As another use case, we have enabled auto-grading of Jupyter Notebook (documents that run in a browser and can contain equations, visualizations, live code, and text) assignments by providing immediate feedback to students and updating scores to the Canvas gradebook. We provide a plugin to upload notebook files to the WebCAT auto-grading system directly from the browser. We integrate Aalto University’s ACOS content (Python and Java exercises) into OpenDSA as a third use case.

Effective software testing identifies potential bugs and helps correct them, producing more reliable and maintainable software. As software development processes have evolved, incremental testing techniques have grown in popularity, particularly with introduction of test-driven development (TDD). However, many programmers struggle to adopt TDD's "test a little, code a little" approach and conventional computer science classrooms neglect evaluating software development as a process. In response, we explore influences on students' testing behaviors, effects of incremental testing strategies, and describe approaches to help computer science students adopt good testing practices. First, to understand students' perspectives and adoption of testing strategies, we investigated their attitudes toward different aspects of TDD. In addition, we observed trends in when and how thoroughly students tested their code and how these choices impacted the quality of their assignments. However, with insight into why students struggle to adopt incremental testing, we identified a need to assess their behaviors during the software development process as a departure from traditional product-oriented evaluation. By building upon an existing automated grading system, we developed an adaptive feedback system to provide customized incentives to reinforce incremental testing behaviors while students solved programming assignments. We investigated how students react to concrete testing goals and hint reward mechanisms and found approaches for identifying testing behaviors and influencing short-term behavioral change. Moreover, we discovered how students incorporate automated feedback systems into their software development strategies. Finally, we compared testing strategies students exhibited through analyzing five years and thousands of snapshots of students' code during development. Even when accounting for factors such as procrastinating on assignments, we found that testing early and consistently maintaining testing throughout development helps produce better quality code and tests. By applying our findings of student software development behaviors to effective testing strategies and teaching techniques, we developed a framework for adaptively scaffolding feedback to empower students to critically reflect over their code and adopt incremental testing approaches. / Ph. D.

Learning technologies in computer science education have been most closely associated with teaching of programming, including automatic assessment of programming exercises. However, when it comes to teaching computer science content and concepts, learning technologies have not been heavily used. Perhaps the best known application today is Algorithm Visualization (AV), of which there are hundreds of examples. AVs tend to focus on presenting the procedural aspects of how a given algorithm works, rather than more conceptual content. There are also new electronic textbooks (eTextbooks) that incorporate the ability to edit and execute program examples. For many traditional courses, a longstanding problem is lack of sufficient practice exercises with feedback to the student. Automated assessment provides a way to increase the number of exercises on which students can receive feedback. Interactive eTextbooks have the potential to make it easy for instructors to introduce both visualizations and practice exercises into their courses. OpenDSA is an interactive eTextbook for data structures and algorithms (DSA) courses. It integrates tutorial content with AVs and automatically assessed interactive exercises. Since Spring 2013, OpenDSA has been regularly used to teach a fundamental data structures and algorithms course (CS2), and also a more advanced data structures, algorithms, and analysis course (CS3) at various institutions of higher education. In this thesis, I report on findings from early adoption of the OpenDSA system. I describe how OpenDSA's design addresses obstacles in the use of AV systems. I identify a wide variety of use for OpenDSA in the classroom. I found that instructors used OpenDSA exercises as graded assignments in all the courses where it was used. Some instructors assigned an OpenDSA assignment before lectures and started spending more time teaching higher-level concepts. OpenDSA also supported implementing a ``flipped classroom'' by some instructors. I found that students are enthusiastic about OpenDSA and voluntarily used the AVs embedded within OpenDSA. Students found OpenDSA beneficial and expressed a preference for a class format that included using OpenDSA as part of the assigned graded work. The relationship between OpenDSA and students' performance was inconclusive, but I found that students with higher grades tend to complete more exercises. / Ph. D.

A Theory of Algorithms course is essential to any Computer Science curriculum at both the undergraduate and graduate levels. It is also considered to be difficult material to teach or to learn. In particular the topics of Computational Complexity Theory, reductions, and the NP-Complete class of problems are considered difficult by students. Numerous algorithm visualizations (AVs) have been developed over the years to portray the dynamic nature of known algorithms commonly taught in undergraduate classes. However, to the best of our knowledge, the instructional material available for NP-Completeness is mostly static and textual, which does little to alleviate the complexity of the topic. Our aim is to improve the pedagogy of NP-Completeness by providing intuitive, interactive, and easy-to-understand visualizations for standard NP Complete problems, reductions, and proofs. In this thesis, we present a set of visualizations that we developed using the OpenDSA framework for certain NP-Complete problems. Our paradigm is a three step process. We first use an AV to illustrate a particular NP-Complete problem. Then we present an exercise to provide a first-hand experience with attempting to solve a problem instance. Finally, we present a visualization of a reduction as a part of the proof for NP-Completeness. Our work has been delivered as a collection of modules in OpenDSA, an interactive eTextbook system developed at Virginia Tech. The tutorial has been introduced as a teaching supplement in both a senior undergraduate and a graduate class. We present an analysis of the system use based on records of online interactions by students who used the tutorial. We also present results from a survey of the students. / Master of Science

Automated grading systems provide feedback to computer science students in a variety of ways, but often focus on incorrect program behaviors. These tools will provide indications of test case failures or runtime errors, but without debugging skills, students often become frus- trated when they don't know where to start. They know their code has defects, but finding the problem may be beyond their experience, especially for beginners. An additional concern is balancing the need to provide enough direction to be useful, without giving the student so much direction that they are effectively given the answer. Drawing on the experiences of the software engineering community, in this work we apply a technique called statistical fault location (SFL) to student program assignments. Using the GZoltar software tool, we applied this technique to a set of previously-submitted student assignments gathered from students in our introductory CS course, CS 1114: Introduction to Software Design. After a manual inspection of the student code, this exercise demonstrated that the SFL technique identifies the defective method in the first three most suspicious methods in the student's code 90% of the time. We then developed a plug-in for Web-CAT to allow new student submissions to be evaluated with the GZoltar SFL system. Additionally, we developed a tool to create a heat map visualization to show the results of the SFL evaluation overlaid on the student's source code. We deployed this toolset for use in CS 1114 in Fall 2017. We then surveyed the students about their perceptions of the utility of the visualization for helping them understand how to find and correct the defects in their code, versus not having access to the heat map. Their responses led to refinements in our presentation of the feedback. We also evaluated the performance of CS 1114 classes from two semesters and discovered that having the heat maps led to more frequent incremental improvements in their code, as well as reaching their highest correctness score on instructor-provided tests more quickly than students that did not have access to the heat maps. Finally, we suggest several directions for future enhancements to the feedback interface. / Doctor of Philosophy / Automated grading systems provide feedback to computer science students in a variety of ways, but often focus on incorrect program behaviors. These tools will provide indications of test case failures or runtime errors, but without debugging skills, students often become frus- trated when they don't know where to start. They know their code has defects, but finding the problem may be beyond their experience, especially for beginners. An additional concern is balancing the need to provide enough direction to be useful, without giving the student so much direction that they are effectively given the answer. Drawing on the experiences of the software engineering community, in this work we apply a technique called statistical fault location (SFL) to student program assignments. Using the GZoltar software tool, we applied this technique to a set of previously-submitted student assignments gathered from students in our introductory CS course, CS 1114: Introduction to Software Design. After a manual inspection of the student code, this exercise demonstrated that the SFL technique identifies the defective method in the first three most suspicious methods in the student's code 90% of the time. We then developed a plug-in for Web-CAT to allow new student submissions to be evaluated with the GZoltar SFL system. Additionally, we developed a tool to create a heat map visualization to show the results of the SFL evaluation overlaid on the student's source code. We deployed this toolset for use in CS 1114 in Fall 2017. We then surveyed the students about their perceptions of the utility of the visualization for helping them understand how to find and correct the defects in their code, versus not having access to the heat map. Their responses led to refinements in our presentation of the feedback. We also evaluated the performance of CS 1114 classes from two semesters and discovered that having the heat maps led to more frequent incremental improvements in their code, as well as reaching their highest correctness score on instructor-provided tests more quickly than students that did not have access to the heat maps. Finally, we suggest several directions for future enhancements to the feedback interface.

Blockchain technologies and cryptocurrency have made a significant impact on today's computing and financial sectors, and the use cases for blockchain applications are increasing day by day. However, there is little understanding of blockchain and cryptocurrencies amongst the general public. In this work, we present LACE, a kinetic sculpture and decentralized ledger created to educate audiences on the complexities of cryptocurrency creation through a visual form. We discuss the design and implementation of LACE as a modular system constructed of 10 kinetic units, each unit containing an array of Microsoft Surface tablets and one delta robot arm to perform touch based operations on each tablet with a modified stylus. Through this structure, we establish a distributed computing system in which each tablet represents blockchain nodes that maintain copies of the blockchain, mine for new blocks and process transactions through visual software interfaces. Additionally, we implement an interactive gaming module to help audiences understand the work of blockchain creation and the mining process. Finally, we evaluate the LACE project's effectiveness to teach audiences through a detailed questionnaire at the 2022 Accelerate Festival in Washington, DC. We found that 73% of visitors agreed they were able to learn something new from LACE and 82% enjoyed their interaction with LACE. / Master of Science / Global technology, computing, economic and financial sectors are all increasingly influenced by the use of the relatively new technologies known as blockchain and cryptocurrency. A blockchain is a publicly distributed digital ledger that keeps track of transaction data securely through cryptography. This technology is heavily associated with the global economy, following the introduction of the cryptocurrency Bitcoin in 2008. Cryptocurrency has often been compared to fiat currencies which are not backed by a commodity with intrinsic value like gold. Bitcoin is seen as a commodity due to its scarcity, with approximately 19 million bitcoins in existence and can be used as a monetary value to purchase goods and services. Studies have shown that a large segment of the general public has little to no understanding of these concepts, even those who have significant related investments. To help expand the understanding of these topics to general audiences, we present LACE; a kinetic sculpture and digital ledger designed to educate audiences on the complexities of cryptocurrency creation through visual and interactive demonstration. LACE demonstrates the processes of blockchain technologies through physical robotic movement and interactive software visualizations. Consisting of a collection of 10 acrylic hexagon units stacked together like building blocks to mimic a virtual network, each unit interacts with an array of Microsoft Surface tablets through an operating robot arm and modified stylus. These tablets illustrate the work of a blockchain through various visualizations, demonstrating the work of nodes and miners who operate and maintain a blockchain network. To help audiences understand the work of blockchain creation and the mining process, we implement an interactive gaming module where participants can act as a miner within a blockchain network and assist in the process of mining for new blocks, help maintain the blockchain and process transactions. We evaluate the LACE project's effectiveness to teach audiences through a detailed questionnaire at the 2022 Accelerate Festival in Washington, DC. We found that 76% of visitors had a better understanding of blockchain concepts following their interactions with LACE and 82% enjoyed their interaction with the sculpture overall.

The fundamentals of software testing and related activities are often elusive in undergraduate curricula. A direct consequence of the lack of software testing efforts during education is the huge losses suffered by the software industry when applications are not sufficiently tested. Software practitioners have exhorted faculty members and institutions to teach more software testing in universities. The purpose of this research is to provide answers to the needs of such practitioners and introduce software-testing activities throughout the curriculum. The most important goal is to introduce software-testing education without requiring a significant amount of extra effort on behalf of faculty members or teaching assistants. The approach taken comprises the development of the Web-based Center for Automated Testing (Web-CAT) and the introduction of test-driven development (TDD) in courses. Web-CAT serves as a learning environment for software testing tasks and helps automatically assess student assignments. A comparison of student programs developed using Web-CAT with historical records indicated a significant decrease in the number of bugs in submitted programs. Undergraduate students also received exposure to the principles of software testing and were able to write test cases that were on an average better than those generated by an automated test case generator designed specifically for the assignment. / Master of Science

Grading is one of the key components of modern pedagogy. Grades are primarily meant to be a method of communication, which is initiated by the owners of the pedagogy (instructors, school, etc.) and are received by various parties such as students: as feedback on their learning outcomes, parents, potential employers, other institutions, etc. However, traditional grading practices in wide use across institutions globally suffer from many problems and can prove to be a hindrance to students achieving their learning outcomes. Many of these practices do not have a backing in education and social research and suffer from various problems such as inherent bias, rewarding of behavior over skill or knowledge, increasing student anxiety, etc. In contrast, EGPs have been backed in research, follow a more open assessment method and have been shown to induce increased learning. Thus, to mitigate some of these problems, we employ Equitable Grading Practices (EGPs) in an introductory CS-1 course at Virginia Tech for the Fall 2023 and Spring 2024 semesters. In this thesis, we introduce these practices and evaluate student perceptions of them thereafter to answer research questions so that we may be able to refine these practices. We gather student experiences primarily through two instruments: a survey and one-on-one interviews. We describe these in detail and evaluate them through the use of statistics as well grounded theory analysis to be able to extract student perceptions of these grading practices. / Master of Science / Grading is a fundamental aspect of education, shaping not only student learning but also their future opportunities. Traditionally, grades are intended to communicate a student's performance to various stakeholders, including parents and potential employers. However, standard grading methods suffer for biases and promote anxiety among students, rather than truly reflecting their knowledge and skills. In an effort to address these issues, our research at Virginia Tech investigates the implementation of Equitable Grading Practices (EGPs) in an introductory computer science course. These practices, supported by educational research, emphasize a more transparent and inclusive approach to assessment, aiming to enhance learning rather than merely evaluate it. This thesis examines how students perceive these new grading practices through surveys and detailed interviews. Our goal is to understand their experiences and refine these methods to better support student achievement and well-being. This study not only offers insights into the impact of EGPs but also contributes to broader educational reforms aimed at fairer and more effective grading systems.

Research has shown that there are gaps in knowledge between newly hired and experienced professionals and that some of these gaps are related to concepts, such as the concepts of object orientation. This problem, and the fact that most computer science majors want to work in the software industry, leads to questions regarding why these gaps exist and how students can be better prepared for their future careers. Against this background, this thesis addresses two theme-based perspectives that focus on students' views of concepts in Computer Science. The first theme-based perspective investigated the existence of potential Threshold Concepts in Computer Science. Such concepts should be troublesome, transformative, irreversible, and integrative. Qualitative methods have been mainly used and empirical data have been collected through semi-structured interviews, concept maps, and written stories. The results identified two Threshold Concepts, suggested several more, and then described the ways in which these concepts have transformed students. The second theme-based perspective took a phenomenographic approach to find the variation in how students understand concepts related to the software profession. Data were collected via semi-structured interviews. In one study the interviews were held in connection with role-playing where students took on the role of a newly hired programmer. The results show a variety of ways to experience the addressed phenomena in the student collective, ranging from superficial views that often have a practical nature to more sophisticated understandings that reflect a holistic approach, including a professional point of view. Educators can use the results to emphasize concepts that are important from students' perspectives. The phenomenographic outcome spaces can help teachers to reflect upon their own ways of seeing contrasted with student conceptions. I have indicated how variation theory can be applied to open more sophisticated ways of seeing, which in this context stresses the professional aspects to help students prepare for becoming professional software developers.