Automated grading systems provide feedback to computer science students in a variety of ways, but often focus on incorrect program behaviors. These tools will provide indications of test case failures or runtime errors, but without debugging skills, students often become frus- trated when they don't know where to start. They know their code has defects, but finding the problem may be beyond their experience, especially for beginners. An additional concern is balancing the need to provide enough direction to be useful, without giving the student so much direction that they are effectively given the answer. Drawing on the experiences of the software engineering community, in this work we apply a technique called statistical fault location (SFL) to student program assignments. Using the GZoltar software tool, we applied this technique to a set of previously-submitted student assignments gathered from students in our introductory CS course, CS 1114: Introduction to Software Design. After a manual inspection of the student code, this exercise demonstrated that the SFL technique identifies the defective method in the first three most suspicious methods in the student's code 90% of the time. We then developed a plug-in for Web-CAT to allow new student submissions to be evaluated with the GZoltar SFL system. Additionally, we developed a tool to create a heat map visualization to show the results of the SFL evaluation overlaid on the student's source code. We deployed this toolset for use in CS 1114 in Fall 2017. We then surveyed the students about their perceptions of the utility of the visualization for helping them understand how to find and correct the defects in their code, versus not having access to the heat map. Their responses led to refinements in our presentation of the feedback. We also evaluated the performance of CS 1114 classes from two semesters and discovered that having the heat maps led to more frequent incremental improvements in their code, as well as reaching their highest correctness score on instructor-provided tests more quickly than students that did not have access to the heat maps. Finally, we suggest several directions for future enhancements to the feedback interface. / Doctor of Philosophy / Automated grading systems provide feedback to computer science students in a variety of ways, but often focus on incorrect program behaviors. These tools will provide indications of test case failures or runtime errors, but without debugging skills, students often become frus- trated when they don't know where to start. They know their code has defects, but finding the problem may be beyond their experience, especially for beginners. An additional concern is balancing the need to provide enough direction to be useful, without giving the student so much direction that they are effectively given the answer. Drawing on the experiences of the software engineering community, in this work we apply a technique called statistical fault location (SFL) to student program assignments. Using the GZoltar software tool, we applied this technique to a set of previously-submitted student assignments gathered from students in our introductory CS course, CS 1114: Introduction to Software Design. After a manual inspection of the student code, this exercise demonstrated that the SFL technique identifies the defective method in the first three most suspicious methods in the student's code 90% of the time. We then developed a plug-in for Web-CAT to allow new student submissions to be evaluated with the GZoltar SFL system. Additionally, we developed a tool to create a heat map visualization to show the results of the SFL evaluation overlaid on the student's source code. We deployed this toolset for use in CS 1114 in Fall 2017. We then surveyed the students about their perceptions of the utility of the visualization for helping them understand how to find and correct the defects in their code, versus not having access to the heat map. Their responses led to refinements in our presentation of the feedback. We also evaluated the performance of CS 1114 classes from two semesters and discovered that having the heat maps led to more frequent incremental improvements in their code, as well as reaching their highest correctness score on instructor-provided tests more quickly than students that did not have access to the heat maps. Finally, we suggest several directions for future enhancements to the feedback interface.

Blockchain technologies and cryptocurrency have made a significant impact on today's computing and financial sectors, and the use cases for blockchain applications are increasing day by day. However, there is little understanding of blockchain and cryptocurrencies amongst the general public. In this work, we present LACE, a kinetic sculpture and decentralized ledger created to educate audiences on the complexities of cryptocurrency creation through a visual form. We discuss the design and implementation of LACE as a modular system constructed of 10 kinetic units, each unit containing an array of Microsoft Surface tablets and one delta robot arm to perform touch based operations on each tablet with a modified stylus. Through this structure, we establish a distributed computing system in which each tablet represents blockchain nodes that maintain copies of the blockchain, mine for new blocks and process transactions through visual software interfaces. Additionally, we implement an interactive gaming module to help audiences understand the work of blockchain creation and the mining process. Finally, we evaluate the LACE project's effectiveness to teach audiences through a detailed questionnaire at the 2022 Accelerate Festival in Washington, DC. We found that 73% of visitors agreed they were able to learn something new from LACE and 82% enjoyed their interaction with LACE. / Master of Science / Global technology, computing, economic and financial sectors are all increasingly influenced by the use of the relatively new technologies known as blockchain and cryptocurrency. A blockchain is a publicly distributed digital ledger that keeps track of transaction data securely through cryptography. This technology is heavily associated with the global economy, following the introduction of the cryptocurrency Bitcoin in 2008. Cryptocurrency has often been compared to fiat currencies which are not backed by a commodity with intrinsic value like gold. Bitcoin is seen as a commodity due to its scarcity, with approximately 19 million bitcoins in existence and can be used as a monetary value to purchase goods and services. Studies have shown that a large segment of the general public has little to no understanding of these concepts, even those who have significant related investments. To help expand the understanding of these topics to general audiences, we present LACE; a kinetic sculpture and digital ledger designed to educate audiences on the complexities of cryptocurrency creation through visual and interactive demonstration. LACE demonstrates the processes of blockchain technologies through physical robotic movement and interactive software visualizations. Consisting of a collection of 10 acrylic hexagon units stacked together like building blocks to mimic a virtual network, each unit interacts with an array of Microsoft Surface tablets through an operating robot arm and modified stylus. These tablets illustrate the work of a blockchain through various visualizations, demonstrating the work of nodes and miners who operate and maintain a blockchain network. To help audiences understand the work of blockchain creation and the mining process, we implement an interactive gaming module where participants can act as a miner within a blockchain network and assist in the process of mining for new blocks, help maintain the blockchain and process transactions. We evaluate the LACE project's effectiveness to teach audiences through a detailed questionnaire at the 2022 Accelerate Festival in Washington, DC. We found that 76% of visitors had a better understanding of blockchain concepts following their interactions with LACE and 82% enjoyed their interaction with the sculpture overall.

The fundamentals of software testing and related activities are often elusive in undergraduate curricula. A direct consequence of the lack of software testing efforts during education is the huge losses suffered by the software industry when applications are not sufficiently tested. Software practitioners have exhorted faculty members and institutions to teach more software testing in universities. The purpose of this research is to provide answers to the needs of such practitioners and introduce software-testing activities throughout the curriculum. The most important goal is to introduce software-testing education without requiring a significant amount of extra effort on behalf of faculty members or teaching assistants. The approach taken comprises the development of the Web-based Center for Automated Testing (Web-CAT) and the introduction of test-driven development (TDD) in courses. Web-CAT serves as a learning environment for software testing tasks and helps automatically assess student assignments. A comparison of student programs developed using Web-CAT with historical records indicated a significant decrease in the number of bugs in submitted programs. Undergraduate students also received exposure to the principles of software testing and were able to write test cases that were on an average better than those generated by an automated test case generator designed specifically for the assignment. / Master of Science

Grading is one of the key components of modern pedagogy. Grades are primarily meant to be a method of communication, which is initiated by the owners of the pedagogy (instructors, school, etc.) and are received by various parties such as students: as feedback on their learning outcomes, parents, potential employers, other institutions, etc. However, traditional grading practices in wide use across institutions globally suffer from many problems and can prove to be a hindrance to students achieving their learning outcomes. Many of these practices do not have a backing in education and social research and suffer from various problems such as inherent bias, rewarding of behavior over skill or knowledge, increasing student anxiety, etc. In contrast, EGPs have been backed in research, follow a more open assessment method and have been shown to induce increased learning. Thus, to mitigate some of these problems, we employ Equitable Grading Practices (EGPs) in an introductory CS-1 course at Virginia Tech for the Fall 2023 and Spring 2024 semesters. In this thesis, we introduce these practices and evaluate student perceptions of them thereafter to answer research questions so that we may be able to refine these practices. We gather student experiences primarily through two instruments: a survey and one-on-one interviews. We describe these in detail and evaluate them through the use of statistics as well grounded theory analysis to be able to extract student perceptions of these grading practices. / Master of Science / Grading is a fundamental aspect of education, shaping not only student learning but also their future opportunities. Traditionally, grades are intended to communicate a student's performance to various stakeholders, including parents and potential employers. However, standard grading methods suffer for biases and promote anxiety among students, rather than truly reflecting their knowledge and skills. In an effort to address these issues, our research at Virginia Tech investigates the implementation of Equitable Grading Practices (EGPs) in an introductory computer science course. These practices, supported by educational research, emphasize a more transparent and inclusive approach to assessment, aiming to enhance learning rather than merely evaluate it. This thesis examines how students perceive these new grading practices through surveys and detailed interviews. Our goal is to understand their experiences and refine these methods to better support student achievement and well-being. This study not only offers insights into the impact of EGPs but also contributes to broader educational reforms aimed at fairer and more effective grading systems.

Large Language Models (LLMs) have emerged as a revolutionary force in Computer Science Education, offering unprecedented opportunities to facilitate learning and comprehension. Their application in the classroom, however, is not without challenges. LLMs are prone to hallucination and contextual inaccuracies. Furthermore, they risk exposing learning processes to cheating illicit practices and providing explicit solutions that impede the development of critical thinking skills in students. To address these pitfalls and investigate how specialized LLMs can enhance engagement among learners particularly using LLMs, we present Gurukul, a unique coding platform incorporating dual features - Retrieval Augmented Generation and Guardrails. Gurukul's practice feature provides a hands-on code editor to solve DSA problems with the help of a dynamically Guardrailed LLM to prevent explicit code solutions. On the other hand, Gurukul's Study feature incorporates a Retrieval Augmented Generation mechanism that uses OpenDSA as its source of truth, allowing the LLM to fetch and present information accurately and relevantly, thereby trying to overcome the issue of inaccuracies. We present these features to evaluate the user perceptions of LLM-assisted educational tools. To evaluate the effectiveness and utility of Gurukul in a real-world educational setting, we conducted a User Study and a User Expert Review with students (n=40) and faculty (n=2), respectively, from a public state university in the US specializing in DSA courses. We examine student's usage patterns and perceptions of the tool and report reflections from instructors and a series of recommendations for classroom use. Our findings suggest that Gurukul had a positive impact on student learning and engagement in learning DSA. This feedback analyzed through qualitative and quantitative methods indicates the promise of the utility of specialized LLMs in enhancing student engagement in DSA learning. / Master of Science / Computer science education is continuously evolving with new technologies enhancing the learning experience. This thesis introduces Gurukul, an innovative platform designed to transform the way students learn Data Structures and Algorithms (DSA). Gurukul integrates large language models (LLMs) with advanced features like Retrieval Augmented Generation (RAG) and Guardrails to create an interactive and adaptive learning environment. Traditional learning methods often struggle with providing accurate information and engaging students actively. Gurukul addresses these issues by offering a live code editor for hands-on practice and a study feature that retrieves accurate information from trusted sources. The platform ensures students receive context-sensitive guidance without bypassing critical thinking skills. A study involving students and faculty from a public university specializing in DSA courses evaluated Gurukul's effectiveness. The feedback, based on qualitative and quantitative evaluations, highlights the platform's potential to enhance student engagement and learning outcomes in computer science education. This research contributes to the ongoing development of educational technologies and provides insights for future improvements.

Research has shown that there are gaps in knowledge between newly hired and experienced professionals and that some of these gaps are related to concepts, such as the concepts of object orientation. This problem, and the fact that most computer science majors want to work in the software industry, leads to questions regarding why these gaps exist and how students can be better prepared for their future careers. Against this background, this thesis addresses two theme-based perspectives that focus on students' views of concepts in Computer Science. The first theme-based perspective investigated the existence of potential Threshold Concepts in Computer Science. Such concepts should be troublesome, transformative, irreversible, and integrative. Qualitative methods have been mainly used and empirical data have been collected through semi-structured interviews, concept maps, and written stories. The results identified two Threshold Concepts, suggested several more, and then described the ways in which these concepts have transformed students. The second theme-based perspective took a phenomenographic approach to find the variation in how students understand concepts related to the software profession. Data were collected via semi-structured interviews. In one study the interviews were held in connection with role-playing where students took on the role of a newly hired programmer. The results show a variety of ways to experience the addressed phenomena in the student collective, ranging from superficial views that often have a practical nature to more sophisticated understandings that reflect a holistic approach, including a professional point of view. Educators can use the results to emphasize concepts that are important from students' perspectives. The phenomenographic outcome spaces can help teachers to reflect upon their own ways of seeing contrasted with student conceptions. I have indicated how variation theory can be applied to open more sophisticated ways of seeing, which in this context stresses the professional aspects to help students prepare for becoming professional software developers.

This thesis explores students' experiences of specialist information technology (IT) courses at the secondary school level in New Zealand. It asks whether students experience a gendered curriculum culture in relation to specialist IT subjects. The exploration involves a survey of national curriculum arrangements and detailed consideration of the manner in which the curriculum is implemented in practice by teachers and experienced by students in three case study classes in a conventional high school, Kahikatea High School (KHS). These classes are year 12 computer studies (CPS) and years 12 and 10 text and information management (TIM). Twenty-two students were the focus of detailed observation in the course of a year. It is found that students experience a gendered IT curriculum culture at KHS, which takes form in both gendered subject and classroom cultures. Gendered subject cultures are established in part through national curriculum structures that maintain subjects from historically gendered domains. Conservative local subject arrangements at KHS contribute to a gendered curriculum in practice. The curriculum takes on a gendered character as a function of choice - teachers' choices about subjects they will offer and the way courses are organised and presented, and students' choices about what subjects they will take. Particular subjects and courses are associated with nominally masculine and feminine computer practices and are thereby imbued with masculine and feminine subject identities. There is considerable variation and nuance in the way students experience different IT courses and in the meanings they make of their experiences. In short, individual students experience the same course differently. They are influenced to greater and lesser degrees by a range of factors, including expectations, prior experience, classroom pedagogy, classroom relationships and performance. Also, individual students are negotiating their masculine and feminine identities as students of IT and computer users as they participate in specialist IT courses and in other arenas of their lives. As they negotiate their roles as computer users and students of IT at KHS, males and females are established in relations of power or authority with the technology and with each other - as computer controllers, aspirant controllers and competent users. These relationships have a gendered character that derives from the attribution of the status of controllers to (some) males and the exclusion of females from this group. However, individual males and females aspire to and are attributed the characteristics and status commensurate with a range of user roles. Gender is a factor in individual students' experiences, but in ways that defy stereotyping and that are highly individualised. All this suggests that gender is not essential in the sense that it implies sameness, but also that gender is not passé or inconsequential as a factor in students experiences of specialist IT courses. Gender relations are a fundamental and inescapable feature of students' experiences of the IT curriculum in practice at KHS.

Teste de software é uma atividade do processo de desenvolvimento de software que busca revelar os defeitos presentes no software. Apesar dos constantes avanços e surgimento de diferentes modelos pedagógicos, este assunto continua sendo ensinado com o modelo tradicional de ensino. A comunidade de educação em computação vem investigando a aplicação de diferentes modelos pedagógicos e, nessa perspectiva, emerge o modelo pedagógico flipped classroom como uma interessante solução. Estudos mencionam sobre a necessidade de investigá-lo, buscando reconhecer seu potencial no ensino de um determinado conteúdo. Além disso, implementar o modelo exige uma intensificação de esforços para o professor, pois ele precisará planejar atividades e mecanismos de apoio ao ensino para cada um dos momentos previstos pelo modelo. Trabalhos relatam que os alunos durante o momento de estudo antes da aula podem ter dificuldade em entender aspectos teóricos sobre o conteúdo, pois esse pode ter conceitos que podem gerar dupla interpretação. Este trabalho de mestrado tem o propósito de contribuir com o ensino de teste de software, investigando o impacto do modelo flipped classroom no ensino de teste de software. O projeto também propõe uma abordagem de apoio ao planejamento e implantação desse modelo e analisa o uso de um agente conversacional como mecanismo de apoio ao flipped classroom. Estudos experimentais foram realizados visando entender o impacto do modelo no processo de aprendizado de teste de software. Na sequência, foram analisados conteúdos de teste de software ensinados em cursos de computação no Brasil, definidos objetivos de aprendizagem para esses conteúdos, selecionados mecanismos de apoio e atividades para contemplar os três momentos previstos pelo modelo e implementado um mecanismo para apoiar a implantação do modelo flipped classroom no ensino de teste. Por fim, um agente conversacional foi proposto e investigado no contexto de uma aula de teste de software em que o modelo flipped classroom foi utilizado. Os resultados deste trabalho indicam que o modelo flipped classroom permite que os alunos tenham melhores resultados no aprendizado, especialmente quando há realização de atividades práticas. Os alunos revelaram que preferem este modelo em detrimento ao tradicional. No entanto, o modelo produz uma carga de trabalho maior para o aluno. Por fim, o uso do agente conversacional como um mecanismo de apoio durante o momento antes da aula indica sua relevância para o flipped classroom. / Software testing is an activity of the development process that aims to reveal the faults present in the software. Despite of constant progress and emergence of different pedagogical models, this subject is still taught with strategies based mainly on the traditional model. Although software testing is considered a relevant subject, some students are not very motivated to learn it. The computer education community has investigated the use of different teaching models and, from this perspective, the Flipped Classroom pedagogical model has emerged as an interesting solution. Several studies mention that is required to investigate this model in order to identify its potential in teaching a specific subject and/or teaching area. In addition, the implementation of this model demands an intensification of efforts from the teacher because he/she needs to plan activities and support mechanisms for all phases of this model: before the class, during the class, and after the class. Results from the literature indicate that students may present difficulties in understand the theory during the phase before the classbecause studying alone can lead to many doubts and misunderstandings. Therefore, this masters thesis investigates the impact of the Flipped Classroom model on teaching of testing techniques and criteria. Experimental studies were initially undertaken aiming to understand the models impact in the learning process of software testing. Following, we analyzed what subjects of software testing are taught in Computer courses in Brazil. Based on the results, we defined learning objectives for these subjects, selected support mechanisms and activities to contemplate the three phases of the Flipped Classroom model. An approach, named FlipClass-STT, to support the implementation of this model was defined and experimentally evaluated. Finally, the conversational agent TOB-STT was proposed and evaluated in context of a software testing class, in which the Flipped Classroom model was employed. The results of this work indicate that Flipped Classroom allows the students to have better learning outcomes, especially when performing practical activities. The students have revealed that they prefer this model against the traditional one. However, it is also observed that model produces a greater workload for the student. The use of TOB-STT how a support mechanism during the phase before the class indicate its relevance and impact for learning software testing.

abstract: Feedback represents a vital component of the learning process and is especially important for Computer Science students. With class sizes that are often large, it can be challenging to provide individualized feedback to students. Consistent, constructive, supportive feedback through a tutoring companion can scaffold the learning process for students. This work contributes to the construction of a tutoring companion designed to provide this feedback to students. It aims to bridge the gap between the messages the compiler delivers, and the support required for a novice student to understand the problem and fix their code. Particularly, it provides support for students learning about recursion in a beginning university Java programming course. Besides also providing affective support, a tutoring companion could be more effective when it is embedded into the environment that the student is already using, instead of an additional tool for the student to learn. The proposed Tutoring Companion is embedded into the Eclipse Integrated Development Environment (IDE). This thesis focuses on the reasoning model for the Tutoring Companion and is developed using the techniques of a neural network. While a student uses the IDE, the Tutoring Companion collects 16 data points, including the presence of certain key words, cyclomatic complexity, and error messages from the compiler, every time it detects an event, such as a run attempt, debug attempt, or a request for help, in the IDE. This data is used as inputs to the neural network. The neural network produces a correlating single output code for the feedback to be provided to the student, which is displayed in the IDE. The effectiveness of the approach is examined among 38 Computer Science students who solve a programming assignment while the Tutoring Companion assists them. Data is collected from these interactions, including all inputs and outputs for the neural network, and students are surveyed regarding their experience. Results suggest that students feel supported while working with the Companion and promising potential for using a neural network with an embedded companion in the future. Challenges in developing an embedded companion are discussed, as well as opportunities for future work. / Dissertation/Thesis / Masters Thesis Software Engineering 2019

Educators and researchers continue to explore the benefits, real or imagined, of implementing pair programming as part of the computer science pedagogy. Current reviews of computer science educational research practices do not focus on educational pair programming. This thesis presents a review of the research methods used in recent educational pair programming research. The primary purpose of this review is to inform the ongoing dialogue about and to provide evidence-based recommendations for improving educational pair programming research. Replicating the design of a previous computer science education methodological review, this study inspected a sample of 108 articles from a population of 129 of articles related to educational pair programming published from 2000 to 2008. Articles were classified using a 112-variable taxonomy, identifying report elements, research methodology, research design, kinds of variables inspected, and statistical practices. Major findings include several differences between the methodological characteristics of educational pair programming research when compared to general computer science education research, including: (a) an increased proportion of studies involving human participants, (b) a decreased proportion of quantitative methodologies, and (c) an increased proportion of controlled research designs. There exists some minor evidence that researchers affiliated with institutions in the United States are more likely than their counterparts outside of the United States to inspect only student attitudes and implement a posttest-only research design, and less likely to implement an experimental or quasi-experimental methodology