No description available.

No description available.

While mutation testing is an effective fault-based testing technique, it has its challenges such as being computationally expensive and requiring a large amount of effort to review surviving mutants. These problems have resulted in mutation testing mostly being restricted to academic research and not as widely adopted in the industry. In the academic context, the focus has been on maximizing the mutation score and while a high mutation score might increase the quality of the software, it is not feasible to kill all the mutants. Moreover, all mutants are not as equally important, and some can not or should not be killed. Instead, the focus should be shifted to prioritizing the productive mutants, those that further improve the test suite or the source code. This thesis investigated if some mutation operators are more suitable for certain types of software by using selective mutation. The mutation operators were evaluated based on their ability to generate productive mutants. Moreover, the mutation operators were analyzed to identify how they could be improved to reduce the number of unproductive mutants generated by them. Dextool Mutate was used to conduct mutation testing on four open-source C/C++ software that were all different types of software. It was concluded that some mutation operators are more suitable for certain types of software resulting in the proposal of a set of mutation operators for each software type. Moreover, various improvements for the mutation operators were identified that reduce the number of unproductive mutants generated. Lastly, it may be helpful to customize the implementation of mutation operators for each type of software and some software types may require additional specialized mutation operators.

Bug Fixer is a web-based application that complements lectures with hands-on exercises that encourage students to think about the logic in programs. Bug Fixer presents students with code that has several bugs that they must fix. The process of fixing the bugs forces students to conceptually think about the code and reinforces their understanding of the logic behind algorithms. In this work, we conducted a study using Bug Fixer with undergraduate students in the CSCE1040 course at University of North Texas to evaluate whether the system increases their conceptual understanding of the algorithms and improves their Software Testing skills. Students participated in weekly activities to fix bugs in code. Most students enjoyed Bug Fixer and recommend the system for future use. Students typically reported a better understanding of the algorithms used in class. We observed a slight increase of passing grades for students who participated in our study compared to students in other sections of the course with the same instructor who did not participate in our study. The students who did not report a positive experience provide comments for future improvements that we plan to address in future work.

Big amounts of test data generated by the current used software testing tools (QA-C/QA-C++ and Cantata) contain a variety of different values. The variances cause enormous challenges in data aggregation and interpretation that directly affect generation of test metrics. Due to the circumstance of data processing, this master thesis introduces a database-centric test management framework for test metrics aims at centrally handling the big data as well as facilitating the generation of test metrics. Each test result will be individually parsed to be a particular format before being stored in a centralized database. A friendly front-end user interface is connected and synchronized with the database that allows authorized users to interact with the stored data. With a granularity tracking mechanism, any stored data will be systematically located and programmatically interpreted by a test metrics generator to create various kinds of high-quality test metrics. The automatization of the framework is driven by Jenkins CI to automatically and periodically performing the sequential operations. The technology greatly and effectively optimizes and reduces effort in the development, as well as enhance the performance of the software testing processes. In this research, the framework is only started at managing the testing processes on software-unit level. However, because of the independence of the database from levels of software testing, it could also be expanded to support software development at any level.

Submitted by Luciana Ferreira (lucgeral@gmail.com) on 2016-10-04T12:46:23Z No. of bitstreams: 2 Dissertação - Bruno Nunes Machado - 2016.pdf: 954291 bytes, checksum: 2b4b0a80a709d8803e7d0857e9aad0dd (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2016-10-04T12:46:59Z (GMT) No. of bitstreams: 2 Dissertação - Bruno Nunes Machado - 2016.pdf: 954291 bytes, checksum: 2b4b0a80a709d8803e7d0857e9aad0dd (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Made available in DSpace on 2016-10-04T12:46:59Z (GMT). No. of bitstreams: 2 Dissertação - Bruno Nunes Machado - 2016.pdf: 954291 bytes, checksum: 2b4b0a80a709d8803e7d0857e9aad0dd (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Previous issue date: 2016-09-01 / Fundação de Amparo à Pesquisa do Estado de Goiás - FAPEG / The software testing is an important component of software development life cycle, that directly affects quality of software products. Some problems in software testing phase can not be optimized only with traditional Software Engineering techniques. It is possible to do the mathematical modelling of those problems in an attempt to optimize them through the search techniques. However, the use of optimization approaches tend to incorporate more and more activities decisions to the tester, making more complex test activity. So, in order that optimization techniques are in fact employed at the Software Test solutions, the ability to abstract the details of optimization are required. Thus, the objective of this research is to propose a framework for search-based software testing (SBST). The proposed framework works as a top-level layer over generic optimization frameworks and testing software tools, it's target is supporting software testers that are not able to use optimization frameworks during a testing activity due to short deadlines and limited resources or skills, also supporting expert or beginners users from optimization area that need or want to compare their metaheuristics with ones from literature and offered by the proposed framework. The framework was evaluated in a case study of software testing scenario. This scenario was modeled as test case selection problem in which experiments were executed with different metaheuristics and benchmarks offered by framework. The results indicate it's capability to support the SBST area with emphasis on the test cases selection. The framework was evaluated and compared with other SBST frameworks in terms of quality metrics, that indicated its extensibility and flexibility as framework. / O Teste de Software é uma parte essencial do processo de desenvolvimento de software, com impacto direto na qualidade do produto de software. Alguns problemas detectados durante a fase de teste de software não são possíveis de serem resolvidos apenas com as técnicas tradicionais da Engenharia de Software. Nestes casos é possível realizar a modelagem matemática desses problemas e tentar otimizá-los por meio das técnicas de busca. Entretanto, a utilização de abordagens de otimização tende a incorporar mais decisões e mais atividades para o testador, tornando a atividade de teste mais complexa. Assim, para que as técnicas de otimização sejam de fato empregadas no Teste de Software, soluções com a capacidade de abstrair detalhes da otimização são necessárias. Diante disso, o objetivo desta pesquisa consiste em propor um framework para apoiar o Teste de Software Baseado em Busca. O framework proposto funciona como uma camada de alto nível sobre os frameworks genéricos de otimização e as ferramentas de teste de software, apoiando testadores de software que não são capazes de utilizar os frameworks de otimização durante uma atividade de teste devido a prazos curtos e recursos ou habilidades limitadas, além de apoiar usuários iniciantes ou especialistas da área de otimização que precisam ou desejam comparar suas metaheurísticas ou heurísticas com as da literatura e as oferecidas pelo framework proposto. O framework foi avaliado em um estudo de caso no cenário de teste de software. Tal cenário foi modelo como um problema de seleção de casos de teste, em que experimentos foram executados com diferentes metaheurísticas e benchmarks oferecidos pelo framework. Os resultados indicaram a capacidade do framework em apoiar a aréa de SBST, com destaque para o problema de seleção de casos de teste. Além disso, o framework também foi avaliado e comparado com outro framework SBST em termos de métricas de qualidade, que indicaram a extensibilidade e flexibilidade do framework proposto.

The aim of this thesis is to improve the reliability and quality of new requested functionality, and existing modules, at Fortum HR System Solutions. This was conducted through an evaluation of the test processes by implementing principles of Software Testing and Test Management. For the study to successfully improve the testing performed at HR System Solutions, existing test processes were analyzed. The analysis was conducted by evaluating the current test processes using theoretical test evaluation styles called maturity models. The methodology of choice was the Testing Maturity Model (TMM), which was adapted to the nature of HR System Solutions requirements, experience and needs. The evaluation used qualitative methods together with pedagogical principles to conduct interviews and workshops to consolidate the theoretical evaluation. Interviews and workshops were conducted within the team of HR System Solutions, where all members of them team contributed to the thesis at some point. An external interview also took place for comparative study. Results of the evaluation, interviews and workshop were compiled and analyzed accordingly. With the analyzed results in place, flaws in the testing processes were apparent. A generalization of the flaws led to the conclusion in the form of a suggestion. The conclusive suggestion was for Fortum HR System Solutions to establish a test committee/group role within the team. Considering the current economical and organizational situation this job role would be a divided job role appointed to current members of the HR System Solutions team. The research creates a walkthrough on a potential method on understanding inefficiencies within testing processes of a company and providing a cause-based solution. / Undersökningen är inriktad på att förbättra tillförlitligheten och kvalitén på ny begärd funktionalitet, och befintliga moduler, på Fortum HR System Solutions. Detta genomfördes genom en utvärdering av testprocesser genom implementation av principer inom Software Testing samt Test Management. För att förbättra testningen som utförs på HR System Solutions var det nödvändigt att analysera de befintliga testprocesserna. Analysen genomfördes genom att utvärdera de nuvarande testprocesserna med hjälp av teoretiska utvärderingsmetoder som kallas Maturity Models. Den valda metoden var Testing Maturity Model (TMM) som tillämpades med avseende på HR System Solutions förutsättningar, erfarenheter och behov. Utvärderingen använder sig av kvalitativa metoder samt pedagogiska principer för att genomföra intervjuer och workshops stärka den teoretiska utvärderingen från TMM. Intervjuer och workshops genomfördes inom HR System Solutions arbetslag, där alla medlemmar bidrog till examensarbetet. Även en extern intervju gjordes i jämförande syfte. Resultatet av utvärderingen sammanställdes och analyserades i enlighet med de förnämnda teoretiska ramvärken. Med hjälp av analyserna visade sig bristerna hos HR System Solutions testprocesser. En generalisering av bristerna gjordes, vilket ledde till slutsatsen i form av ett förslag. Det slutliga förslaget innebar att Fortum HR System Solutions behöver upprätta en testkommitté/grupp inom arbetslaget. Undersökningen visar en potentiell metod för att förstå ineffektiviteten inom testprocesser i ett företag, och tillhandahålla en orsaks-baserad lösning.

<p>    </p> <p>Irregular control-flow structures like deeply nested conditional branches are common in real-world software applications. Improving the performance and efficiency of such programs is often challenging because it is difficult to analyze and optimize programs with irregular control flow. We observe that real-world programs contain similar or identical computations within different code paths of the conditional branches. Compilers can merge similar code to improve performance or code size. However, existing compiler optimizations like code hoisting/sinking, and tail merging do not fully exploit this opportunity. We propose a new technique called Control-Flow Melding (CFM) that can merge similar code sequences at the control-flow region level. We evaluate CFM in two applications. First, we show that CFM reduces the control divergence in GPU programs and improves the performance. Second, we apply CFM to CPU programs and show its effectiveness in reducing code size without sacrificing performance. In the next part of this dissertation, we investigate how CFM can be extended to improve dynamic test generation techniques like Dynamic Symbolic Execution (DSE). DSE suffers from path explosion problem when many conditional branches are present in the program. We propose a non-semantics-preserving branch elimination transformation called CFM-SE that reduces the number of symbolic branches in a program. We also provide a framework for detecting and reasoning about false positive bugs that might be added to the program by non-semantics-preserving transformations like CFM-SE. Furthermore, we evaluate CFM-SE on real-world applications and show its effectiveness in improving DSE performance and code coverage. </p>

Thesis (MSc) -- Stellenbosch University, 2003. / ENGLISH ABSTRACT: Software testing tools simplify and automate the menial work associated with testing. Moreover, for complex concurrent software such as computer protocols, testing tools allow testing on an abstract level that is independent of specific implementations. Standard conformance testing methodologies and a number of testing tools are commercially available, but detailed descriptions of the implementation of such testing tools are not widely available. This thesis investigates the development of a tool for automated protocol testing in the ETH Oberon development environment. The need to develop a protocol testing tool that automates the execution of specified test cases was identified in collaboration with a local company that develops protocols in the programming language Oberon. Oberon is a strongly typed secure language that supports modularisation and promotes a readable programming style. The required tool should translate specified test cases into executable test code supported by a runtime environment. A test case consists of a sequence of input actions to which the software under test is expected to respond by executing observable output actions. A number of issues are considered of which the first is concerned with the representation of test case specifications. For this, a notation was used that is basically a subset of the test specification language TTCN-3 as standardised by the European Telecommunications Standards Institute. The second issue is the format of executable test cases and a suitable runtime environment. A translator was developed that generates executable Oberon code from specified test cases. The compiled test code is supported by a runtime library, which is part of the tool. Due to the concurrent nature of a protocol environment, concurrent processes in the runtime environment are identified. Since ETH Oberon supports multitasking in a limited sense, test cases are executed as cooperating background tasks. The third issue is concerned with the interaction between an executing test case and a system under test. It is addressed by an implementation dependent interface that maps specified test interactions onto real interactions as required by the test context in which an implementation under test operates. A supporting protocol to access the service boundary of an implementation under test remotely and underlying protocol service providers are part of a test context. The ETH Oberon system provides a platform that simplifies the implementation of protocol test systems, due to its size and simple task mechanism. Operating system functionality considered as essential is pointed out in general terms since other systems could be used to support such testing tools. In conclusion, directions for future work are proposed. / AFRIKAANSE OPSOMMING: Toetsstelsels vir programmatuur vereenvoudig en outomatiseer die slaafse werk wat met toetsing assosieer word. 'n Toetsstelsel laat verder toe dat komplekse gelyklopende programmatuur, soos rekenaarprotokolle, op 'n abstrakte vlak getoets word, wat onafhanklik van spesifieke implementasies is. Daar bestaan standaard metodes vir konformeringstoetsing en 'n aantal toetsstelsels is kommersiëel beskikbaar. Uitvoerige beskrywings van die implementering van sulke stelsels is egter nie algemeen beskikbaar nie. Hierdie tesis ondersoek die ontwikkeling van 'n stelsel vir outomatiese toetsing van protokolle in die ontwikkelingsomgewing van ETH Oberon. Die behoefte om 'n protokoltoetsstelsel te ontwikkel, wat die uitvoering van gespesifiseerde toetsgevalle outomatiseer, is geïdentifiseer in oorleg met 'n plaaslike maatskappy wat protokolle ontwikkel in die Oberon programmeertaal. Oberon is 'n sterkgetipeerde taal wat modularisering ondersteun en a leesbare programmeerstyl bevorder. Die toestsstelsel moet gespesifiseerde toetsgevalle vertaal na uitvoerbare toetskode wat ondersteun word deur 'n looptydomgewing. 'n Toetsgeval bestaan uit 'n reeks van toevoeraksies waarop verwag word dat die programmatuur wat getoets word, sal reageer deur die uitvoering van afvoeraksies wat waargeneem kan word. 'n Aantal kwessies word aangeraak, waarvan die eerste te make het met die voorstelling van die spesifikasie van toetsgevalle. Hiervoor is 'n notasie gebruik wat in wese 'n subversameling van die toetsspesifikasietaal TTCN-3 is. TTCN-3 is gestandardiseer deur die European Telecommunications Standards Institute. Die tweede kwessie is die formaat van uitvoerbare toetsgevalle en 'n geskikte looptydomgewing. 'n Vertaler is ontwikkel wat uitvoerbare Oberon-kode genereer vanaf gespesifiseerde toetsgevalle. Die vertaalde toetskode word ondersteun deur 'n biblioteek van looptydfunksies, wat deel van die stelsel is. As gevolg van die eienskap dat 'n protokolomgewing uit gelyklopende prosesse bestaan, word daar verskillende tipes van gelyklopende prosesse in 'n protokoltoetsstelsel geïdentifiseer. Aangesien ETH Oberon 'n beperkte multitaakstelsel is, word toetsgevalle vertaal na eindige outomate wat uitgevoer word as samewerkende agtergrondtake. Die derde kwessie het te make met die interaksie tussen 'n toetsgeval wat uitgevoer word en die stelsel wat getoets word. Dit word aangespreek deur 'n koppelvlak wat gespesifiseerde interaksies afbeeld op werklike interaksies soos vereis deur die konteks waarin 'n implementasie onderworpe aan toetsing uitvoer. 'n Ondersteunende protokolom die dienskoppelvlak van die implementasie oor 'n afstand te bereik en ander onderliggende protokoldienste is deel van 'n toetskonteks. Die ETH Oberon-stelsel help in die vereenvoudiging van die implementasie van protokol toetsstelsels, as gevolg van die stelsel se grootte en die eenvoudige taakhanteerder . Die essensiële funksionaliteit van bedryfsstelsels word uitgelig in algemene terme omdat ander stelsels gebruik kan word om toetsstelsels te ondersteun. Ten slotte word voorstelle vir opvolgwerk gemaak.

The Software Reliability group at KTH-CSC has designed and built a novel test platform LBTest for black-box requirements testing of reactive and embedded software systems (e.g. web servers, automobile control units, etc). The main concept of LBTest is to create a large number of test cases by incorporation of an automata learning algorithm with a model checking algorithm (NuSMV). This platform aims to support different learned automata, learning algorithms and different model checking algorithms which can be combined to implement the paradigm of learning-based testing (LBT).This thesis project investigates an existing published algorithm for learning deterministic finite automata (DFA)known as Kearns algorithm. The aimof this thesis is to investigate how effective Kearns algorithm is from a software testing perspective.Angluin’s well-known L* DFA learning algorithm has a simple structure and implementation. On the other hand, Kearnsalgorithm has more complex, difficult structure and harder implementation than L* algorithm, however it is more efficient and faster. For this reason, the plan is to implement an advanced DFA learning algorithm, Kearns algorithm[4], from a description in the literature (using Java).We consider a methodology to compare Kearns algorithm with Angluin’s DFA learning algorithm based on the master thesis of Czerny[8].The comparisonsbetween the Kearns and the L* algorithmsare based on the number of membership and equivalence queriesto investigate the difficulty of learning