Regression testing is an expensive software engineering activity intended to provide
confidence that modifications to a software system have not introduced faults.
Test case prioritization techniques help to reduce regression testing cost by ordering
test cases in a way that better achieves testing objectives. In this thesis, we are interested
in prioritizing to maximize a test suite's rate of fault detection, measured by a
metric, APED, trying to detect regression faults as early as possible during testing.
In previous work, several prioritization techniques using low-level code coverage
information had been developed. These techniques try to maximize APED over
a sequence of software releases, not targeting a particular release. These techniques'
effectiveness was empirically evaluated.
We present a larger set of prioritization techniques that use information at arbitrary
granularity levels and incorporate modification information, targeting prioritization
at a particular software release. Our empirical studies show significant
improvements in the rate of fault detection over randomly ordered test suites.
Previous work on prioritization assumed uniform test costs and fault seventies,
which might not be realistic in many practical cases. We present a new cost-cognizant
metric, APFD[subscript c], and prioritization techniques, together with approaches
for measuring and estimating these costs. Our empirical studies evaluate prioritization
in a cost-cognizant environment.
Prioritization techniques have been developed independently with little consideration
of their similarities. We present a general prioritization framework that allows
us to express existing prioritization techniques by a framework algorithm using
parameters and specific functions.
Previous research assumed that prioritization was always beneficial if it improves
the APFD metric. We introduce a prioritization cost-benefit model that more
accurately captures relevant cost and benefit factors, and allows practitioners to assess
whether it is economical to employ prioritization.
Prioritization effectiveness varies across programs, versions, and test suites. We
empirically investigate several of these factors on substantial software systems and
present a classification-tree-based predictor that can help select the most appropriate
prioritization technique in advance.
Together, these results improve our understanding of test case prioritization and
of the processes by which it is performed. / Graduation date: 2004
| Identifer | oai:union.ndltd.org:ORGSU/oai:ir.library.oregonstate.edu:1957/30746 |
| Date | 19 June 2003 |
| Creators | Malishevsky, Alexey Grigorievich |
| Contributors | Rothermel, Gregg E. |
| Source Sets | Oregon State University |
| Language | en_US |
| Detected Language | English |
| Type | Thesis/Dissertation |
Page generated in 0.0019 seconds