Submitted by Fabio Sobreira Campos da Costa (fabio.sobreira@ufpe.br) on 2016-03-15T15:21:11Z
No. of bitstreams: 2
license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5)
TESE_SABRINA.pdf: 1152470 bytes, checksum: a89ffc94cb3ee813cf52ca2c043171ba (MD5) / Made available in DSpace on 2016-03-15T15:21:11Z (GMT). No. of bitstreams: 2
license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5)
TESE_SABRINA.pdf: 1152470 bytes, checksum: a89ffc94cb3ee813cf52ca2c043171ba (MD5)
Previous issue date: 2015-03-31 / Software Product Lines (SPLs) allow engineers to systematically build families of
software products, defined by a unique combination of features—increments in functionality,
improving both the efficiency of the software development process and the quality of the software
developed. However, testing these kinds of systems is challenging, as it may require running each
test against a combinatorial number of products. We call this problem the High Dimensionality
Problem. Another obstacle to product line testing is the absence of Feature Models (FMs),
making it difficult to discover the real causes for test failures. We call this problem the Lack of
Feature Model Problem.
The High Dimensionality Problem is associated to the large space of possible configurations
that an SPL can reach. If an SPL has n boolean features, for example, there are 2n
possible feature combinations. Therefore, systematically testing this kind of system may require
running each test against all those combinations, in the worst case. The Lack of Feature Model
Problem is related to the absence of feature models. The FM enables accurate categorization of
failing tests as failures of programs or the tests themselves, not as failures due to inconsistent
combinations of features. For this reason, the lack of FM presents a huge challenge to discover
the true causes for test failures.
Aiming to solve these problems, we propose two lightweight techniques: SPLat and
SPLif. SPLat is a new approach to dynamically prune irrelevant configurations: the configurations
to run for a test can be determined during test execution by monitoring accesses to
configuration variables. As a result, SPLat reduces the number of configurations. Consequently,
SPLat is lightweight compared to prior works that used static analysis and heavyweight dynamic
execution. SPLif is a technique for testing SPLs that does not require a priori availability of
feature models. Our insight is to use a profile of passing and failing test runs to quickly identify
test failures that are indicative of a problem (in test or code) as opposed to a manifestation of
execution against an inconsistent combination of features.
Experimental results show that SPLat effectively identifies relevant configurations with
a low overhead. We also apply SPLat on two large configurable systems (Groupon and GCC),
and it scaled without much engineering effort. Experimental results demonstrate that SPLif
is useful and effective to quickly find tests that fail on consistent configurations, regardless of
how complete the FMs are. Furthermore, we evaluated SPLif on one large extensively tested
configurable system, GCC, where it helped to reveal 5 new bugs, 3 of which have been fixed
after our bug reports. / Software Product Lines (SPLs) allow engineers to systematically build families of
software products, defined by a unique combination of features—increments in functionality,
improving both the efficiency of the software development process and the quality of the software
developed. However, testing these kinds of systems is challenging, as it may require running each
test against a combinatorial number of products. We call this problem the High Dimensionality
Problem. Another obstacle to product line testing is the absence of Feature Models (FMs),
making it difficult to discover the real causes for test failures. We call this problem the Lack of
Feature Model Problem.
The High Dimensionality Problem is associated to the large space of possible configurations
that an SPL can reach. If an SPL has n boolean features, for example, there are 2n
possible feature combinations. Therefore, systematically testing this kind of system may require
running each test against all those combinations, in the worst case. The Lack of Feature Model
Problem is related to the absence of feature models. The FM enables accurate categorization of
failing tests as failures of programs or the tests themselves, not as failures due to inconsistent
combinations of features. For this reason, the lack of FM presents a huge challenge to discover
the true causes for test failures.
Aiming to solve these problems, we propose two lightweight techniques: SPLat and
SPLif. SPLat is a new approach to dynamically prune irrelevant configurations: the configurations
to run for a test can be determined during test execution by monitoring accesses to
configuration variables. As a result, SPLat reduces the number of configurations. Consequently,
SPLat is lightweight compared to prior works that used static analysis and heavyweight dynamic
execution. SPLif is a technique for testing SPLs that does not require a priori availability of
feature models. Our insight is to use a profile of passing and failing test runs to quickly identify
test failures that are indicative of a problem (in test or code) as opposed to a manifestation of
execution against an inconsistent combination of features.
Experimental results show that SPLat effectively identifies relevant configurations with
a low overhead. We also apply SPLat on two large configurable systems (Groupon and GCC),
and it scaled without much engineering effort. Experimental results demonstrate that SPLif
is useful and effective to quickly find tests that fail on consistent configurations, regardless of
how complete the FMs are. Furthermore, we evaluated SPLif on one large extensively tested
configurable system, GCC, where it helped to reveal 5 new bugs, 3 of which have been fixed
after our bug reports.
| Identifer | oai:union.ndltd.org:IBICT/oai:repositorio.ufpe.br:123456789/15975 |
| Date | 31 March 2015 |
| Creators | SOUTO, Sabrina de Figueirêdo |
| Contributors | http://lattes.cnpq.br/3762670242328435, D'AMORIM, Marcelo Bezerra |
| Publisher | Universidade Federal de Pernambuco, Programa de Pos Graduacao em Ciencia da Computacao, UFPE, Brasil |
| Source Sets | IBICT Brazilian ETDs |
| Language | Portuguese |
| Detected Language | English |
| Type | info:eu-repo/semantics/publishedVersion, info:eu-repo/semantics/doctoralThesis |
| Source | reponame:Repositório Institucional da UFPE, instname:Universidade Federal de Pernambuco, instacron:UFPE |
| Rights | Attribution-NonCommercial-NoDerivs 3.0 Brazil, http://creativecommons.org/licenses/by-nc-nd/3.0/br/, info:eu-repo/semantics/openAccess |
Page generated in 0.003 seconds