University of North Florida. School of Computing
Understanding and augmenting scientific software testing practices
Abstract
dc:description.abstractScientific software can be broadly defined as software written to aid in conducting scientific research activities. As this software is often written by the scientists themselves, who often learned programming to build that software, some important aspects of software development may get less attention than they would have gotten on enterprise software developed by programmers. One area of development that often gets overlooked in this regard is testing. Thus, in this study, we seek to better understand the testing processes and propose methods to improve the effectiveness of those tests. We begin with examining the software's Continuous Integration/Delivery practices. This is primarily done through GitHub Actions, which can be used to automate testing. Based on this, we examined the GitHub Actions of several scientific software practices of various sizes and scientific domains. Results of our study show that, even among the variety in the software, many aspects of their GitHub Actions Workflows are shared. We then conducted a study to evaluate the fault detection effectiveness of the tests in scientific software. Our results show that there do exist common deficiencies in testing among this scientific software, such as low if-statement branch coverage or low edge case sensitivity and lack of thorough testing of mathematical calculations. These shortcomings can potentially be improved through known testing methods. Finally, Based on our findings, we propose and explore a testing method called metamorphic testing, which makes use of necessary properties of a function that identify relationships between a that function's input and its output. We added metamorphic tests to the software's test suites, and found as much as a 10\% improvement on mutation testing results among some types of mutations. The results imply that the improvements made by metamorphic testing can be even further improved when written by someone with domain knowledge.
Degree
thesis:*- Name thesis:degree_name
- Master of Science in Computer and Information Sciences (MS)
- Discipline thesis:degree_discipline
- Computing
- Grantor
- University of North Florida. School of Computing
- Date dc:date.issued
- Fall
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Roker, Kris
- Roker, Kris, 1998-
- Advisors dc:contributor.advisor
-
- Kanewala, Upulee
- Umapathy, Karthikeyan
- Ghosh, Anirban
- Contributors dc:contributor
-
- Dr. Upulee Kanewala
- Dr. Karthikeyan Umapathy
- Dr. Anirban Ghosh
Subjects
dc:subject × 12- Thesis
- University of North Florida
- UNF
- Dissertations, Academic -- UNF -- Master of Science in Computer and Information Sciences
- Dissertations, Academic -- UNF -- Computing
- Scientific Software
- Mutation Testing
- Metamorphic Testing
- Quality Assurance
- Software Development
- Computer Sciences
- Science -- Research -- Software -- Testing; Software failures -- Testing -- Evaluation; Science -- Software -- Testing -- Evaluation; Application software -- Development -- Testing; Computer software -- Development -- Testing; Enterprise application integration (Computer systems) -- Testing
Rights
dc:rightsIdentifiers
dc:identifier.*- Repository record dc:identifier
- https://digitalcommons.unf.edu/etd/1308