Creating a Computer Science and Software Engineering Department that supports students with diverse identities and backgrounds is essential to creating a computing workforce that reflects the world at large. Inspired by the work of Metcalf et al.'s survey conducted at the University of Illinois, we use the same methods to examine the state of our computing department with respect to issues of inclusive climate and student sense of belonging, which have been shown to be important for retention in STEM fields. We use the four areas that contribute to belonging based on the work of Rainey et al. along with a fifth category of learning environment in order to assess our students' sense of belonging. This paper's main focus is based on results from two surveys of Cal Poly Students conducted exactly one year apart (2019: n=154, 2020: n=122). Both surveys were sent to all computing majors in Spring quarter, the last quarter of the regular academic year. We found that 58-68\% of students felt they were not typical computer scientists, which mirrors the results of the survey conducted at the University of Illinois, indicating that the lack of belonging is perhaps a ubiquitous problem within the field of computing. Other salient results include identifying the presence of statistically significant differences for some groups based on gender and race & ethnicity. These differences were found when looking at students' senses of their science identity and learning environment. We also found that women had a significantly greater chance of having strong interpersonal relationships within computing. The survey results are augmented by a survey of first-quarter freshmen in Fall 2019 (n=44) and student interviews conducted in Spring 2021 (n=15). We hope that the addition of these results explain and expand upon our main results and add insight as to how the student experience can evolve from a student's first quarter onward. These differences shine an important light on some positive trends as well as several concerning differences to be examined in our quest to create a diverse and equitable department.

Thesis advisor: Michael George Barnett / Computer science literacy is the key to surviving and thriving in the digital era. Unfortunately, given the negative stereotypes about who does computer science related work and what such work entails, many individuals are dissuaded from learning more about computer science and lack belief in their competence in computer science. As such, this dissertation aims to identify ways to make computer science education more self-efficacious using three connected studies, including (1) a mixed methods study on an intervention project for non-STEM major college students, (2) a practitioner study on a novel curriculum for middle school students, and (3) a study on the internal structure of a novel concept inventory for AI concepts. Findings from the first study confirm the importance of providing learners with mastery experiences in terms of helping them developing self-efficacy in coding. Findings from the second study provide teachers with teaching tips they could use while teaching the AI curriculum in their classrooms. Findings from the third study reveal the strengths and weaknesses of the AI concept inventory in accurately measuring respondents’ knowledge about AI. / Thesis (PhD) — Boston College, 2022. / Submitted to: Boston College. Lynch School of Education. / Discipline: Teacher Education, Special Education, Curriculum and Instruction.

As computer science students develop more complex programs at the end of their first year of course work, comprehending the complex and varied interactions of program execution, potential control flow and data relationships become more and more difficult. Additionally, for instructors when evaluating student's programs, a simplified view of more complex (longer) programs is desirable. This thesis explores algorithms to create a tool for students that provides a simplified view of these concepts via visualization. The tool created for this thesis provides interactive visual representations of student programs. This allows for a simplified representation of the entire program along with depth exploration options to examine potential control flow and data access/mutations. The following is an exploration of program visualization, with a focus on usability in an educational setting. Two main approaches will be discussed. The first attempts to visualize the call graph of a running program by showing what methods call what other methods, and the frequency in which they are invoked. The second shows all potential paths through a non-running program, by viewing the program on a per-method level. This approach also includes information about how methods interact with data. As a test case this thesis focuses on a spell check program which builds a binary search tree dictionary then searches it for input strings and provides correction suggestions if the input is not found in the dictionary. We present an evaluation of our tool via creating visualizations of four different student implementations of this program. These visualizations are then analyzed by computer science faculty to identify common threads throughout all submissions, as well as areas where individual students struggled or excelled. Additionally visualizations are used as a tool in a lecture instructing students about binary search trees. The students provide feedback as to the effectiveness of the visualizations and their comprehension of the material. We conclude that program visualization is a difficult task, especially when students are unused to visualizing control flow. Results indicate potential for use as both a student and instructor tool, though further research is required to identify optimal usage.

According to Fossati and Guzdail, many CS instructors rely on their intuition, feedback from students and anecdotal advice from other instructors to make course improvements. Guzdail noted that: "Computing educators' practice would dramatically improve if we drew on evidence, rather than intuition". This means that Computer Science instructors may benefit from processes and tools that help them make informed changes to their curriculum. An evidence-based approach to course improvement is curriculum alignment, which evaluates the degree which the learning objectives, course content, and assessment methods are in agreement with each other. This provides instructors with a detailed view of their course and areas that need improvement. Current alignment processes are impractical for a course instructor to apply, requiring a panel of experts and training on the process. In this research, I developed a computer-assisted alignment process (CAAP) that uses the concept of traceability from software engineering, to define a process that is applicable by a single course instructor limiting the need for a panel of experts. In an initial application CAAP took 75 hours to apply, consequently a prototype alignment tool (AlignET) was designed to automates the new alignment process providing instructors with results they can use to make course improvements. I evaluated the practicality of AlignET by conducting collective case studies with four participants. Observations and interviews were used to collect data. AlignET reduced the time to complete CAAP to less than 11 hours and the participants identified course improvements, gaps in their instructional methods, and learning objectives they emphasized more than others. The findings from the case study presented key improvements to AlignET. / Doctor of Philosophy / According to Fossati and Guzdail, many CS instructors rely on their intuition, feedback from students and anecdotal advice from other instructors to make course improvements. Guzdail noted that: "Computing educators' practice would dramatically improve if we drew on evidence, rather than intuition". This means that Computer Science instructors may benefit from processes and tools that help them make informed changes to their curriculum. An evidence-based approach to course improvement is curriculum alignment, which evaluates the degree which the learning objectives, course content, and assessment methods are in agreement with each other. This provides instructors with a detailed view of their course and areas that need improvement. Current alignment processes are impractical for a course instructor to apply, requiring a panel of experts and training on the process. In this research, I developed a computer-assisted alignment process (CAAP) that uses the concept of traceability from software engineering, to define a process that is applicable by a single course instructor limiting the need for a panel of experts. In an initial application CAAP took 75 hours to apply, consequently a prototype alignment tool (AlignET) was designed to automates the new alignment process providing instructors with results they can use to make course improvements. I evaluated the practicality of AlignET by conducting collective case studies with four participants. Observations and interviews were used to collect data. AlignET reduced the time to complete CAAP to less than 11 hours and the participants identified course improvements, gaps in their instructional methods, and learning objectives they emphasized more than others. The findings from the case study presented key improvements to AlignET.

Computer Science fields have a difficult time engaging underrepresented populations such as African Americans and women. EarSketch is an approach to engage these student through authentic STEAM learning involving computational music remixing. EarSketch has been used in several pilot studies. In this study, students from one pilot study participated in a focus group to understand the effectiveness of EarSketch in engaging underrepresented minorities. Qualitative analysis shows a variety of contributing factors in engagement such as motivation, confidence, identity, conceptualization, and creativity.

Multimedia content, e.g., sound files, interactive demos, and video files, has been widely used in digital courses to provide an easy to use format and to emphasize the ideas. In this work, we address aspects of generating multimedia contents automatically in digital courses. In particular, we focus on two types of automatically generated multimedia: interactive glossaries and sound files. Glossaries play a major role in enhancing students' comprehension of the course core concepts. Glossary terms have complex interrelationship that cannot be fully illustrated by standard approaches, e.g., including all the terms as a linear, alphabetized list. To overcome this limitation, we introduce an interactive design for the glossary terms using concept maps. Glossary terms are visualized as nodes in graphs and their relationships are included on the edges. We implement these concept maps within the OpenDSA e-textbook system. A concept map associated with the selected term is generated on demand. We evaluate the effectiveness of our design by comparing student use of our concept-map based glossary to the traditional alphabetized list. We have designed new exercises that target the comprehension of the glossary terms to make students familiar with the concept maps. Our other work generates sound files automatically to supplement text narration in slide shows. This is made feasible by the widespread availability of text-to-speech generators in web browsers. To this end, we designed an interactive narration tool and integrated it into the OpenDSA library. In this way, all slide shows automatically have their text augmented with narration. / Master of Science / Recently, there has been an increase in the use of multimedia contents in digital courses. Multimedia files, e.g., sound files, interactive demos, and video files, are used in digital courses to provide an easy to use format and to emphasize the ideas. In this work, we address aspects of generating multimedia contents in digital courses. In particular, we focus on two types of automatically generated multimedia: interactive glossaries and sound files. Glossaries play a major role in enhancing students’ comprehension of the core concepts in the courses. In general, glossary terms have complex interrelationship that cannot be fully illustrated by standard approaches, such as the alphabetized list. To overcome this limitation, we introduce an interactive design for the glossary terms using concept maps. In this design, glossary terms are visualized as nodes in graphs and their relationships are included on the edges. We implement these concept maps within the OpenDSA e-textbook system to be generated on demand. We evaluate the effectiveness of our design by comparing student use of our concept-map based glossary to the traditional alphabetized list. We have designed new exercises to make students familiar with the concept maps. Our other work generates sound files automatically to supplement text narration in slide shows. This was motivated by the widespread use of text-to-speech generators in web browsers. To this end, we designed an interactive narration tool and integrated it into the OpenDSA library so that all OpenDSA slide shows can benefit from the narration tool.

Automatic assessment tools are increasingly utilized in undergraduate programming courses to evaluate software solutions, streamlining the grading process for both students and professors. In spite of their benefits of speed and convenience, such online systems for providing instant feedback have the tendency to draw attention to performance-based outcomes while failing to reliably recognize the effort and hard work a student puts into a solution. For the many struggling students who are new to programming, this type of objective feedback can be discouraging and may decrease their motivation to stay engaged towards success. To address this issue, this paper explores strategies for more effectively recognizing student progress on programming assignments and identifying small tasks for students to complete that will steer them in the right direction. Further, this paper will discuss a gamification approach for adding ``Daily Missions" to Web-CAT, the Web-based Center for Automated Testing. From an evaluation of results gathered from student experiences with this latest version of Web-CAT at Virginia Tech, this paper will highlight the valuable potential gamification has in boosting student engagement in computer science learning. / Master of Science / In college computer science courses today, many students submit their assignments and receive instant feedback through online websites. Although they streamline the grading process for both students and professors, these systems have the tendency to draw attention to performance-based outcomes while failing to reliably recognize the effort and hard work a student puts into a solution. For the many struggling students who are new to programming, this type of objective feedback can be discouraging and may decrease their motivation to stay engaged towards success. To address this issue, this paper explores strategies for more effectively recognizing student progress on programming assignments and identifying small tasks for students to complete that will steer them in the right direction. One strategy in particular is called gamification, which refers to giving game-like attributes to a non-game system. This paper will discuss a gamification approach for adding “Daily Missions” to WebCAT, the Web-based Center for Automated Testing. From an evaluation of results gathered from student experiences with this latest version of Web-CAT at Virginia Tech, this paper will highlight the valuable potential gamification has in boosting student engagement in computer science learning.

Teaching good software development practices is difficult, both in theory and in practice. Time management and project organization are skills often left by the wayside by students too focused on the coding itself. Educational research has been invested in developing strategies to combat these bad habits. In order to provide better support for interventions discouraging bad development habits, more data about student development is needed. The purpose of this research is to design and implement software to collect data continuously as students work on programming projects and provide it in useful forms to instructors and researchers so that they may make headway in designing new curricula, assignments, and interventions that better help students to succeed. The DevEventTracker is a software system that interfaces with existing Web-CAT services to track student development data continuously, without any student effort. Development and compilation events are tracked within the Eclipse IDE through a plugin and sent to a Web-CAT server. Code snapshots corresponding to each event are also committed to a server-side repository. The system provides a dashboard as a set of instructor-visible web pages that display useful data in generated charts and tables. Data are presented in both class overview and individual student summaries. The system presented will enable future research in education and specifically in intervention development. Particularly, the system can be used to allow instructors to identify students who have a tendency to procrastinate and design more effective interventions. / Master of Science

Pair programming is a software development method which describes two programmers working together on the same computer, sharing one keyboard. This approach requires programmers to communicate frequently, which can lead the pair to experience certain benefits over solo programming, such as faster problem solving and a greater enjoyment of their work (Cockburn and Williams, 2001, Bryant et al., 2006). Many programmers approach their first pairing experience with scepticism, having doubts about their partner’s working habits and programming style, and about the additional communication aspects that this programming style entails (Williams et al., 2000). Despite a significant amount of research into pair programming of over 15 years, it is not evident what communication between the pair contributes to the task of pair programming. This work presents an analytic coding scheme which was derived from the observation of the communication of expert pairs working in industry. Over 35 hours of communication across 11 different pairs was analysed. This coding scheme was further refined to produce industry-inspired pair programming guidelines that assist novice pair programmers to improve their experience of pair communication. Findings indicate that introducing these guidelines to novice student pairs can have a positive impact on their perception of intra-pair communication, and on their perception of their partner’s contribution. Feedback received from expert pairs was used to add detail to the guidelines, which have been made publically available through an online resource.

With advances in mobile phone hardware, the demand for mobile applica- tions has risen drastically. This has resulted in mobile phones becoming a pop- ular new medium for application development. However, the body of knowledge for contextual examples and tutorials leaves much to be desired. As of January 2010, California Polytechnic State University has offered a mobile development class that teaches students how to write applications for phones running Google’s Android platform. This class aims at taking advantage of students’ current in- terest in mobile applications to teach them about difficult computer science topics. As a corollary, the class hopes to foster and encourage a sense of inde- pendence and entrepreneurship through having students design, implement, and publish their own applications to the Android Application Marketplace. The main contribution of this thesis project comes in the form of a series of detailed educational laboratory exercises and a system for grading student submissions in an automated fashion. These labs are designed to supplement the Android documentation by providing contextual examples, activities, and tutorials. It is therefore the goal of this thesis project to aid in transforming the class of mobile development students into a group of successful, practicing, mobile developers.