Actuellement, le développement d’un logiciel de taille industriel repose généralement surdes tests ou des preuves unitaires pour garantir rigoureusement ses exigences. En outre, il adéjà été montré que l’utilisation combinée du test et de la preuve unitaires est plus efficaceque l’utilisation d’une seule de ces deux techniques. Néanmoins, un ingénieur en vérificationhésite encore à utiliser ces deux techniques conjointement, faute d’une notion de couverturecommune au test et à la preuve. Définir une telle notion est l’objet de cette thèse.En effet, nous introduisons une nouvelle couverture, appelée « couverture label-mutant ».Elle permet de représenter les critères de couverture structurelle habituels du test, comme lacouverture des instructions, la couverture des branches ou la couverture MC/DC et de décidersi le critère choisi est satisfait en utilisant une technique de vérification formelle, qu’elle soitpar test, par preuve ou par une combinaison des deux. Elle permet également de représenterles critères de couverture fonctionnelle. Nous introduisons aussi dans cette thèse une méthodereposant sur des outils automatiques de test et de preuve pour réduire l’effort de vérificationtout en satisfaisant le critère de couverture choisi. Cette méthode est mise en oeuvre au seinde la plateforme d’analyse de code C (Frama-C), fournissant ainsi à un ingénieur un moyenopérationnel pour contrôler et réaliser la vérification qu’il souhaite. / Currently, industrial-strength software development usually relies on unit testing or unitproof in order to ensure high-level requirements. Combining these techniques has already beendemonstrated more effective than using one of them alone. The verification engineer is yetnot been to combine these techniques because of the lack of a common notion of coverage fortesting and proving. Defining such a notion is the main objective of this thesis.We introduce here a new notion of coverage, named « label-mutant coverage ». It subsumesmost existing structural coverage criteria for unit testing, including statement coverage,branch coverage or MC/DC coverage, while allowing to decide whether the chosen criterionis satisfied by relying on a formal verification technique, either testing or proving or both.It also subsumes functional coverage criteria. Furthermore, we also introduce a method thatmakes use of automatic tools for testing or proving in order to reduce the verification costwhile satisfying the chosen coverage criterion. This method is implemented inside Frama-C, aframework for verification of C code (Frama-C). This way, it offers to the engineer a way tocontrol and to perform the expected verifications.

Unit testing plays a crucial role in ensuring the quality and reliability of software systems. However, manual testing can often be a slow and time-consuming process. With current advancements in artificial intelligence (AI), new tools have emerged for automated unit testing to address this issue. But how do these new AI tools compare to conventional automated unit test generation tools? To answer this question, we compared two state-of-the-art conventional unit test tools (EVOSUITE and RANDOOP) with the sole commercially available AI-based unit test tool (DIFFBLUE COVER) for Java. We tested them on 10 sample classes from 3 real-life projects provided by the Defects4J dataset to evaluate their performance regarding code coverage, mutation score, and fault detection. The results showed that EVOSUITE achieved the highest code coverage, averaging 89%, while RANDOOP and DIFFBLUE COVER achieved similar results, averaging 63%. In terms of mutation score, DIFFBLUE COVER had the lowest average score of 40%, while EVOSUITE and RANDOOP scored 67% and 50%, respectively. For fault detection, EVOSUITE and RANDOOP detected a higher number of bugs (7 out of 10 and 5 out of 10, respectively) compared to DIFFBLUE COVER, which found only 4 out of 10. Although the AI-based tool was outperformed in all three criteria, it still shows promise by being able to achieve adequate results, in some cases even surpassing the conventional tools while generating a significantly smaller number of total assertions and more comprehensive tests. Nonetheless, the study acknowledges its limitations in terms of the restricted number of AI-based tools used and the small number of projects utilized from Defects4J.

Test coverage is an important parameter of analyzing how well the product is being tested in any domain within the IT industry. Unit testing is one of the important processes that have gained even more popularity with the rise in Test driven development (TDD) culture.This degree project, conducted at NASDAQ Technology AB, analyzes the existing unit tests in one of the products, and compares various coverage models in terms of quality. Further, the study examines the factors that affect code coverage, presents the best practices for unit testing, and a proven test process used in a real world project.To conclude, recommendations are given to NASDAQ based on the findings of this study and industry standards. / Testtäckning är en viktig parameter för att analysera hur väl en produkt är testad inom alla domäner i IT-industrin. Enhetstestning är en av de viktiga processerna som har ökat sin popularitet med testdriven utveckling. Detta examensarbete, utfört på NASDAQ Technology AB, analyserar de befintliga testen i en av produkterna, och jämför olika kvalitetsmodeller. Vidare undersöker undersökningen de faktorer som påverkar koddekning, presenterar de bästa metoderna för enhetstestning och en beprövad testprocess som används i ett verkligt världsprojekt. Avslutningsvis ges rekommendationer till NASDAQ baserat på resultaten från denna studie och industristandarder.

International Telemetering Conference Proceedings / October 17-20, 1988 / Riviera Hotel, Las Vegas, Nevada / Software development is becoming less an art form and more an engineering discipline. Methods of software development which leave as little as possible to chance are constantly being sought and documented. However, the gap between what is written and what is actually applied is usually quite wide. The only way this gap can be narrowed is through practical application of these very detailed and complex methods. Since it is unlikely that the complexity of these methods will be reduced, automation must be employed wherever possible in the software development process. This paper addresses the successful development of software for the Navy's Realtime Telemetry Processing System III (RTPS III) using practical application of existing methodology in conjunction with a Computer Aided Software Engineering (CASE) tool. Based on this experience, the conclusion presents implications affecting software development the 1990s.

Para sistematizar os testes e contornar as restrições de tempo e custo associadas à atividade de teste, diversas técnicas, critérios e ferramentas têm sido desenvolvidas. Além disso, visando ao estabelecimento de uma estratégia de teste incremental, que apresente baixo custo de aplicação e alta eficácia em revelar a presença de erros, estudos teóricos e empíricos vêm sendo conduzidos pela comunidade de teste. O presente trabalho está inserido nesse contexto e tem como objetivo a realização de estudos empíricos para comparar a adequação entre os critérios baseados em erros - Análise de Mutantes (teste de unidade) e Mutação de Interface (teste de integração) - visando ao estabelecimento de estratégias de teste de baixo custo e eficazes, que englobem todo o ciclo de desenvolvimento de software. Nessa perspectiva, algumas estratégias incrementais de aplicação dos operadores de mutação de unidade e de integração são definidas, explorando o aspecto complementar dos critérios baseados em mutação, reduzindo com isso os custos da atividade de teste durante as fases do teste de unidade e de integração, sem comprometer sua qualidade. Ainda, um conjunto essencial de operadores de mutação para o critério Mutação de Interface é apresentado. / Techniques, criteria and tools have been developed and investigated making the testing activity more systematic and aiming at overcoming associated time and cost constraints. Pursuing the establishment of an incremental, low-cost and effective testing strategy, theoretical and empirical studies have been conducted by the testing community. The work proposed here is within this context and aims to conduct empirical studies for evaluating the adequacy between error based criteria - Mutation Analysis (unit testing) and Interface Mutation (integration testing). Therefore, this work intends to establish low-cost and effective testing strategies that would comprise all software development cycle. In this perspective, some incremental testing strategies for mutant operators application are defined exploring the complementary aspects of unit and integration error based criteria, reducing theirs costs without losses in testing quality. In this scope, an essential mutant operators set for Interface Mutation criterion is characterized.

Submitted by Liliane Ferreira (ljuvencia30@gmail.com) on 2018-04-12T14:43:28Z No. of bitstreams: 2 Tese - Marllos Paiva Prado - 2018.pdf: 5172064 bytes, checksum: f0f497f5cd15e3d7e789fb5792aa6254 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2018-04-13T11:12:00Z (GMT) No. of bitstreams: 2 Tese - Marllos Paiva Prado - 2018.pdf: 5172064 bytes, checksum: f0f497f5cd15e3d7e789fb5792aa6254 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Made available in DSpace on 2018-04-13T11:12:00Z (GMT). No. of bitstreams: 2 Tese - Marllos Paiva Prado - 2018.pdf: 5172064 bytes, checksum: f0f497f5cd15e3d7e789fb5792aa6254 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Previous issue date: 2018-03-16 / Fundação de Amparo à Pesquisa do Estado de Goiás - FAPEG / Conselho Nacional de Pesquisa e Desenvolvimento Científico e Tecnológico - CNPq / Unit testing is an important activity for improving software quality. Over the years, numerous automated tools have been proposed by the testing research community to enhance this activity. However, this thesis' literature review revealed that several research efforts have not considered the human aspects in the proposal of such tools. Also, unit test practitioners are not having the support of the existing tools to solve some mental tasks associated with the activity. Motivated by this gap, this thesis describes a sequence of studies carried out with the purpose of understanding, characterizing and proposing improvements in the cognitive support provided by the test tools, considering a qualitative approach centered on the perspective of test professionals that work at the unit level. The results revealed some primary tasks that require cognitive support of the tools in unit test review practice, including monitoring of pending and executed unit test tasks and navigation between unit testing artifacts. A framework summarizes the results of this study. A research agenda is developed based on the framework and serves as an actionable instrument for the testing community. The contributions of this study include suggestions for practical improvements to current tools and describe new research opportunities in the topic. Also, the methods used in the research are explained in details. / O teste unitário é uma importante atividade para a melhoria da qualidade do software. Ao longo dos anos, inúmeras ferramentas automatizadas foram propostas pela comunidade de pesquisa de teste para melhorar esta atividade. Contudo, a revisão da literatura de testes atual permite observar que esses esforços de pesquisa não têm considerado os aspectos humanos na proposição dessas ferramentas. Observa-se ainda que os praticantes de teste unitário não têm suporte do ferramental existente para resolução de algumas tarefas mentais associadas à atividade. Considerando-se esta lacuna, esta tese descreve uma sequência de estudos realizados com o intuito de entender, caracterizar e propor melhorias no suporte cognitivo provido pelas ferramentas de teste unitário. Tendo em vista a falta de estudos sobre suporte cognitivo para teste de software, empregou-se uma abordagem qualitativa e centrada na perspectiva dos profissionais de teste que atuam no nível unitário. Os resultados revelaram algumas tarefas primárias que requerem suporte cognitivo das ferramentas na prática de revisão de testes unitário, incluindo o monitoramento de tarefas de teste unitário pendentes e executadas e a navegação entre os artefatos relacionados às unidades testadas. Os resultados são resumidos em um framework e, com base nisso, é desenvolvida uma agenda de pesquisa como instrumento acionável para a comunidade de teste. As contribuições desta tese incluem sugestões de melhorias práticas para as ferramentas atuais e descrevem novas oportunidades de pesquisa no tema. Além disso, são explicados em detalhes os métodos utilizados nesta pesquisa.

In traditional software development, bug detection or testing comes as an afterthought. However, bugs are difficult to detect in the later stages of software development that result in long debugging time. Usually, bugs are left out because of higher concentration on development effort, leaving lesser time for testing. Test-Driven Development (TDD) is a software development process that can reduce the debugging time by catching most of the bugs during development. The development is carried on in small and repeated steps based on test cases. However, TDD is designed to work for object-oriented languages. This thesis investigates the suitability of TDD for non-Object Oriented (OO) languages such as C. TDD can be used with C language with the help of stubbing concept. This thesis also evolves the concepts of stubs and mocks in TDD to be used with C to solve dependency related problems. Finally this thesis analyses some frameworks for TDD in C and provides the trade-offs between them.

Amending old projects of legacy code to include agile practices such as extensive unit testing and refactoring has proven difficult. Since automated unit testing was not widely used a decade ago, much code has been written without unit testing in mind. This is especially true for C++ where RAII has been the dominant pattern .This has resulted in a lot of code that suffers from what best can be described as low testability. This also strongly impedes the creation of new unit tests to already existing code. Due to the lack of unit tests, refactoring is done sparsely and with great reluctance. This thesis work tries to remedy that and, in the scope of a limited case study on an existing code base, looks into different ways of creating and utilizing object seams in legacy C++ code to decouple dependencies to make isolated testing possible. This regards to: What are the impediments for code to be testable in an isolated setting? What are the steps for refactoring code to a testable state? The results can be summarized as to contain a list of factors affecting testability, among them: the use of asserts, global state, object instantiation, work in constructor and breaking Law of Demeter. Further with regards to patterns for refactoring code to a testable state, two types of patterns have crystallized: the injection of dependencies and the masking of dependencies using various techniques. The effect these two base patterns have on breaking dependencies on the base level and the Meta level is outlined. A catalogue of patterns has been compiled as part of the appendix.   Inversion of Control (IoC) is a principle used to decoupling classes and since strong dependences is often an attribute giving grievances with regard to testability, it was a central concern in this thesis. IoC can be simplified from a developer standpoint with the help of frameworks or what is usually referred to as IoC containers. Two IoC containers for C++ were evaluated: Autumn Framework PocoCapsule In the evaluation of the two IoC containers it was concluded that Autumn was not mature enough as of the time of the evaluation to be used in production setting. PocoCapsule, even though compelling for some of its powerful features with regard to DSM and HOT, its configuration sometimes require in-code workarounds, affecting its usability in some set of scenarios. However, the big difference was with regard to how the two containers approaches configuration. PocoCapsule uses static analysis of its XML configuration file making it truly declarative while Autumn does runtime parsing and dynamic invocations resulting in a situation closer to procedural scripting.

Para sistematizar os testes e contornar as restrições de tempo e custo associadas à atividade de teste, diversas técnicas, critérios e ferramentas têm sido desenvolvidas. Além disso, visando ao estabelecimento de uma estratégia de teste incremental, que apresente baixo custo de aplicação e alta eficácia em revelar a presença de erros, estudos teóricos e empíricos vêm sendo conduzidos pela comunidade de teste. O presente trabalho está inserido nesse contexto e tem como objetivo a realização de estudos empíricos para comparar a adequação entre os critérios baseados em erros - Análise de Mutantes (teste de unidade) e Mutação de Interface (teste de integração) - visando ao estabelecimento de estratégias de teste de baixo custo e eficazes, que englobem todo o ciclo de desenvolvimento de software. Nessa perspectiva, algumas estratégias incrementais de aplicação dos operadores de mutação de unidade e de integração são definidas, explorando o aspecto complementar dos critérios baseados em mutação, reduzindo com isso os custos da atividade de teste durante as fases do teste de unidade e de integração, sem comprometer sua qualidade. Ainda, um conjunto essencial de operadores de mutação para o critério Mutação de Interface é apresentado. / Techniques, criteria and tools have been developed and investigated making the testing activity more systematic and aiming at overcoming associated time and cost constraints. Pursuing the establishment of an incremental, low-cost and effective testing strategy, theoretical and empirical studies have been conducted by the testing community. The work proposed here is within this context and aims to conduct empirical studies for evaluating the adequacy between error based criteria - Mutation Analysis (unit testing) and Interface Mutation (integration testing). Therefore, this work intends to establish low-cost and effective testing strategies that would comprise all software development cycle. In this perspective, some incremental testing strategies for mutant operators application are defined exploring the complementary aspects of unit and integration error based criteria, reducing theirs costs without losses in testing quality. In this scope, an essential mutant operators set for Interface Mutation criterion is characterized.

This master's thesis describes the possibility of using a continues integration and testing in real company Logio. The result of thesis is implementation of continuous integration tools Jenkins and specific testing for the company Logio. The theoretical part deals with basic principles, techniques and tools defining continuous integration. Subsequently are describing existing tools designed for PHP programming language. The thesis also analyzes various types and possibilities of application testing. Also explains the problems in product development in company Logio, which are consequence of the need for introducing continues integration techniques. The practical part of thesis describes the implementation of a test framework PwTester and its use in continuous integration. In conclusion, the results are evaluated and outlined the possibility of extension.