Mutation testing and adequacy testing are two major technologies to assure the quality of software. In this thesis, we present the first work that alleviates the high cost of mutation testing and ineffectiveness of adequacy testing for context-aware applications. We also present large-scale multi-subject case studies to evaluate how our work successfully alleviates these problems. Mutation testing incurs a high execution cost if randomly selected test inputs kill a small percentage of remaining live mutants. To address this problem, we formulate the notion of context diversity to measure the context changes inherent in test inputs, and propose three context-aware strategies in the selection of test inputs. The empirical results show that the use of test inputs with higher context diversity can significantly benefit mutation testing in terms of resulting in fewer test runs, fewer test case trials, and smaller resultant test suites that achieve a high mutation score level. The case study also shows that at the test case level, the context diversity of test inputs positively and strongly correlates with multiple types of adequacy metrics, which provide a foundation on why context diversity contributes to the effectiveness of test cases in revealing faults in context-aware applications. In adequacy testing, many strategies randomly select test cases to construct adequate test suites with respect to program-based adequacy criteria. They usually exclude redundant test cases that are unable to improve the coverage of the test requirements of an adequacy criterion achieved by constructing test suites. These strategies have not explored in the diversity in test inputs to improve the test effectiveness of test suites. To address this problem, we propose three context-aware refined strategies to check whether redundant test cases can replace previously selected test cases to achieve the same coverage level but with different context diversity levels. The empirical study shows that context diversity can be significantly injected into adequate test suites, and favoring test cases with higher context diversity can significantly improve the fault detection rates of adequate test suites for testing context-aware applications. In conclusion, this thesis makes the significant contributions to the research in testing context-aware applications: (1) It has formulated context diversity, a novel metric to measure context changes inherent in test inputs. (2) It has proposed three context-aware strategies to select test cases with different levels of context diversity. Compared with the baseline strategy, the strategy CAS-H that uses test cases with higher context diversity can significantly reduce the cost of mutation testing over context-aware applications in terms of less number of test runs, smaller adequate test suites, and less number of test inputs used to construct test suites. (3) It has defined three context-aware refined strategies to construct adequate test suites with different context diversity levels. Compared with the baseline strategy, the strategy CARS-H that favors test cases with higher context diversity can significantly improve the effectiveness of adequacy testing in terms of higher fault detection rates. / published_or_final_version / Computer Science / Doctoral / Doctor of Philosophy

Software testing is an important factor of the software development process. Integration testing is an important and expensive level of the software testing process. Unfortunately, since the developers have limited time to perform integration testing and debugging and integration testing becomes very hard as the combinations grow in size, the chain of calls from one module to another grow in number, length, and complexity. This research is about providing new methodology for integration testing to reduce the number of test cases needed to a significant degree while returning as much of its effectiveness as possible. The proposed approach shows the best order in which to integrate the classes currently available for integration and the external method calls that should be tested and in their order for maximum effectiveness. Our approach limits the number of integration test cases. The integration test cases number depends mainly on the dependency among modules and on the number of the integrated classes in the application. The dependency among modules is determined by using an information retrieval technique called Latent Semantic Indexing (LSI). In addition, this research extends the mutation testing for use in integration testing as a method to evaluate the effectiveness of the integration testing process. We have developed a set of integration mutation operators to support development of integration mutation testing. We have conducted experiments based on ten Java applications. To evaluate the proposed methodology, we have created mutants using new mutation operators that exercise the integration testing. Our experiments show that the test cases killed more than 60% of the created mutants.