Software testing is a tedious and very expensive undertaking. Automatic test data generation is, therefore, proposed in this research to help testers reduce their work as well as ascertain software quality. The concept of test driven development (TDD) has become increasingly popular during the past several years. According to TDD, test data should be prepared before the beginning of code implementation. Therefore, this research asserts that the test data should be generated from the software design documents which are normally created prior to software code implementation. Among such design documents, the UML state machine diagrams are selected as a platform for the proposed automated test data generation mechanism. Such diagrams are selected because they show behaviours of a single object in the system. The genetic algorithm (GA) based approach has been developed and applied in the process of searching for the right amount of quality test data. Finally, the generated test data have been used together with UML class diagrams for JUnit test code generation. The GA-based test data generation methods have been enhanced to take care of parallel path and loop problems of the UML state machines. In addition the proposed GA-based approach is also targeted to solve the diagrams with parameterised triggers. As a result, the proposed framework generates test data from the basic state machine diagram and the basic class diagram without any additional nonstandard information, while most other approaches require additional information or the generation of test data from other formal languages. The transition coverage values for the introduced approach here are also high; therefore, the generated test data can cover most of the behaviour of the system.

The main part in the testing of the software is in the generation of test cases suitable for software system testing. The quality of the test cases plays a major role in reducing the time of software system testing and subsequently reduces the cost. The test cases, in model de- sign stages, are used to detect the faults before implementing it. This early detection offers more flexibility to correct the faults in early stages rather than latter ones. The best of these tests, that covers both static and dynamic software system model specifications, is one of the chal- lenges in the software testing. The static and dynamic specifications could be represented efficiently by Unified Modelling Language (UML) class diagram and sequence diagram. The work in this thesis shows that High Level Petri Nets (HLPN) can represent both of them in one model. Using a proper model in the representation of the software specifications is essential to generate proper test cases. The research presented in this thesis introduces novel and automated test cases generation techniques that can be used within a software sys- tem design testing. Furthermore, this research introduces e cient au- tomated technique to generate a formal software system model (HLPN) from semi-formal models (UML diagrams). The work in this thesis con- sists of four stages: (1) generating test cases from class diagram and Object Constraint Language (OCL) that can be used for testing the software system static specifications (the structure) (2) combining class diagram, sequence diagram and OCL to generate test cases able to cover both static and dynamic specifications (3) generating HLPN automat- ically from single or multi sequence diagrams (4) generating test cases from HLPN. The test cases that are generated in this work covered the structural and behavioural of the software system model. In first two phases of this work, the class diagram and sequence diagram are decomposed to nodes (edges) which are linked by Classes Hierarchy Table (CHu) and Edges Relationships Table (ERT) as well. The linking process based on the classes and edges relationships. The relationships of the software system components have been controlled by consistency checking technique, and the detection of these relationships has been automated. The test cases were generated based on these interrelationships. These test cases have been reduced to a minimum number and the best test case has been selected in every stage. The degree of similarity between test cases is used to ignore the similar test cases in order to avoid the redundancy. The transformation from UML sequence diagram (s) to HLPN facilitates the simpli cation of software system model and introduces formal model rather than semi-formal one. After decomposing the sequence diagram to Combined Fragments, the proposed technique converts each Combined Fragment to the corresponding block in HLPN. These blocks are con- nected together in Combined Fragments Net (CFN) to construct the the HLPN model. The experimentations with the proposed techniques show the effectiveness of these techniques in covering most of the software system specifications.

Finding a way to automate the generation of test data is a crucial aspect of software testing. Testing comprises 50% of all software development costs [Korel90]. Finding a way to automate testing would greatly reduce cost and labor involved in the task of software testing. One of the ways to automate software testing is to automate the generation of test data inputs. For example, in statement coverage, creating test cases that will cover all of the conditions required when testing that program would be costly and time-consuming if undertaken manually. Therefore, a way must be found that allows the automation of creating test data inputs to satisfy all test requirements for a given test. One such way of automating test data generation is the use of genetic algorithms. Genetic algorithms use the creation of generations of test inputs, and then choose the most fit test inputs, or those test inputs that are most likely to satisfy the test requirement, as the test inputs that will be passed to the next generation of inputs. In this way, the solution to the test requirement problem can be found in an evolutionary fashion. Current research suggests that comparison of genetic algorithms with random test input generation produces varied results. While results of these studies show promise for the future use of genetic algorithms as an answer to the issue of discovering test inputs that will satisfy branch coverage, what is needed is additional experimental research that will validate the performance of genetic algorithms in a test environment. This thesis makes use of the EvoSuite plugin tool, which is a software plugin for the IntelliJ IDEA Integrated Development Environment that runs using a genetic algorithm as its main component. The EvoSuite tool is run against 22 Java classes, and the EvoSuite tool will automatically generate unit tests and will also execute those unit tests while simultaneously measuring branch coverage of the unit tests against the Java classes under test. The results of this thesis’ experimental research are that, just as the literature indicates, the EvoSuite tool performed with varied results. In particular, Fraser’s study of the EvoSuite tool as an Eclipse plugin was accurate in depicting how the EvoSuite tool would come to perform as an IntelliJ plugin, namely that the EvoSuite tool would perform poorly for a large number of classes tested.

This thesis deals with software process improvement based on the deployment package Software testing, which is a guide for implementation of testing related processes of the international standard ISO/IEC 29110 Life-Cycle Profiles for Very Small Entities. The aim of this thesis is localization of the deployment package Software testing into Czech and realization of the pilot project based on this package on the real software project in a very small enterprise. Another aim is to evaluate the deployment package and to propose modifications to improve it. The real benefit of this thesis is in realization of the pilot project itself, which resulted in testing process improvement in the enterprise. Another expected benefits of this thesis based on its goals are in actual localization of the deployment package into Czech, its evaluation and suggestion of its modifications. Thesis as a whole can be used as a guide and information source for other enterprises seeking an improvement of the testing process as it provides an example of the deployment package Software testing implementation and also theoretical introduction to testing and software process improvement in a small enterprise.

Este trabalho descreve a Klein, uma biblioteca genérica para álgebra linear em C++. A Klein facilita o uso de matrizes e vetores, permitindo que o usuário programe de modo similar ao Matlab. Com ela podemos, por exemplo, implementar um passo do método de Newton para a função f, através da expressão x = x - inv(jac(x)) * f(x), onde x é o vetor, jac a Jacobiana e inv a inversa. Além disso, por se tratar de uma biblioteca genérica, os tipos envolvidos nestas expressões podem ser escolhidos pelo programador. O trabalho também discute como a biblioteca é testada, tanto do ponto de vista de corretude quanto de desempenho. / We describe the Klein library, a generic libray for linear algebra in C++. It simplifies the use of vectors and matrices and let the user program as in Matlab. With Klein, one can for instance implement Newton\'s method as x = x - inv(jac(x)) * f(x), where x is a vector, jac is the Jacobian matrix, inv is the inverse operator and f(x) is the function of which we want to find zero. Moreover, Klein is generic in the sense that it allows the use of arbitrary types of scalars (float, double, intervals, rationals, etc). We also explain how it is tested, both for correctness and performance.

Diversas pesquisas apoiam e investigam o teste de programas concorrentes, as quais objetivam, principalmente, a proposição de critérios de teste e mecanismos para execução das diferentes sincronizações entre processos ou threads. As características específicas dessas aplicações podem ocasionar diferentes tipos de defeitos, os quais, em sua maioria, não são facilmente identificados. Nesse contexto, a geração automática de dados de teste pode apoiar a atividade de teste atuando na seleção de entradas mais representativas, ou seja, aquelas com maior probabilidade de revelar defeitos. Apesar disso, poucas pesquisas abordam este tema no contexto de programas concorrentes, e as existentes não consideram aspectos importantes desse tipo de aplicação. A geração de dados de teste para programas sequenciais dispõe de uma variedade de técnicas que apoiam a seleção dos dados de teste. Essas técnicas têm sido estendidas para o contexto de programas concorrentes partindo da premissa que esses programas necessitam de abordagens mais complexas para seleção de entradas, em decorrência disso um maior custo é imposto ao teste. Considerando esse contexto, uma lacuna ainda em aberto é a avaliação das técnicas para o cenário de programas concorrentes. Neste trabalho a avaliação das técnicas foi explorada por meio da realização de estudos experimentais, os quais avaliaram diferentes técnicas de geração de dados de teste para o contexto de programas concorrentes, considerando a eficácia em revelar defeitos, cobertura de critérios e custo para atividade de teste. Os resultados obtidos demonstraram que as técnicas empregadas para programas sequenciais não atingem o mínimo esperado para este tipo aplicação. Apesar disso, as técnicas investigadas apresentaram características importantes que podem auxiliar a atividade de teste para programas concorrentes e a proposição de abordagens efetivas de geração de dados para esse contexto. / The concurrent program testing has been largely investigated with propositions of testing criteria and mechanisms, which aim mainly to testing criteria proposition and mechanisms for execution of different synchronizations. The specific characteristics of these applications can lead to the different types of faults, which, in most of cases, are not easily identified. In this context, the automatic test data generation can support the testing activity acting in selecting the most representative data tests, i.e. those most likely to reveal faults. Nevertheless, few studies address this issue in the context of concurrent programs, and the these studies do not consider important aspects of this type of application. In contrast, we can find several techniques proposed to support the test data generation for sequential programs. These techniques have been extended to the context of concurrent programs on the premise that these programs require more complex approaches for selection of test data. As a result, a greater cost for testing activity is enforced. Considering this context, a gap still open is the evaluation of techniques for the scenario of concurrent programs. In this work the evaluation of techniques was explored through experimental studies, which different techniques of test data generatoon were evaluated, considering effectiveness, testing coverage and application cost. The results showed that the generation techniques used for sequential programs used in the experimental study do not reach the minimum expected in terms of effectiveness and cost for concurrent programs. Nevertheless, the techniques investigated showed significant features that can help the proposition of effective approaches for test data generation applied to concurrent programs.

A dependência tecnológica da sociedade atual e a flexibilidade necessária para constantes mudanças exigem produtos de software corretos e disponíveis para uso. Os testes são atividades intrínsecas à produção de software e ultimamente vêm ganhando importância como um negócio empresarial próprio, independente do desenvolvimento e com perspectivas de crescimento. Essa atividade é substancialmente prestação de serviço, uma vez que entrega qualidade a respeito de um produto de software e cuja avaliação só pode ser feita de acordo com parâmetros de julgamento do cliente, além de considerar a presença e ingerência do cliente em boa parte do processo de teste. O Brasil é reconhecidamente um dos mais importantes centros de excelência em testes de software e tem se distinguido também na execução de forma independente, o que tem provocado rearranjo de papéis, responsabilidades e forças na atuação da cadeia de valor do software. Este trabalho procura responder como se organizam as operações das empresas de testes independentes de software (ETIS) de forma a torná-las singulares e específicas. Grande parte das pesquisas nesse tema têm se concentrado em questões técnicas de como modelar bons testes de software, registrando-se pouca reflexão sobre temas organizacionais nessa área. Para tanto, foi elaborado um modelo conceitual de organização das operações, capaz de revelar a singularidade da atividade empresarial. O modelo foi submetido a avaliação por meio de um estudo de casos múltiplos, envolvendo seis ETIS e também organizações brasileiras clientes e desenvolvedoras de software. A pesquisa revelou importantes práticas empresariais das ETIS. Destacam-se os seguintes pontos: perfil empresarial empreendedor e inovador, preocupação na entrega de serviço contendo o maior quantidade possível de não-conformidades, constituição de um portfólio diversificado e substantivo de projetos de teste, integração de sistemas produtivos tipo fábrica e personalizado e uso cada vez maior de automação nos processos. A experiência daqueles que vêm utilizando os serviços das ETIS tem comprovado um aumento efetivo no nível de qualidade do software e o mercado, por sua vez, tem refinado os critérios de seleção das ETIS, exigindo-lhes alta maturidade. / Technology dependence of modern society and the flexibility to accommodate constant changes require correct and ready software products. Testing activities are intrinsic to the software production, which lately have been gaining importance as an independent business and broad prospects for growth. This activity is substantially service because it deliveries software product quality which evaluation can only be made according to client\'s trial parameters, besides considering their presence and interference in much of the testing process. Brazil is recognized as an excellence center leading in software testing issues and has also distinguished itself in independent performance, which has caused rearrangement of roles, responsibilities, actions and powers in the software value chain relationships. This paper attempts to answer how to organize the independent software testing enterprises (ETIS) business operations, in order to make this business model unique and specific. Software testing research has focused more on technical issues, especially about how to model and specify good software testing and less about organizational issues. This research aims a conceptual model of organizational operations, able to reveal the uniqueness of business activity. The proposal was subjected to evaluation using a multiple case study involving six ETIS and also other Brazilian organizations as customers and software developers. The research revealed important business practices of ETIS, which have ensured the success of the enterprise and business profitability. It was highlight the following points: entrepreneurial and innovative business profile, high concern in service delivery as software product quality, establishment of a portfolio of diverse and substantive testing projects, factory and custom productive systems integration and increasing use of automation in testing processes. The experience of those who have used ETIS services has proven an effective increase in the level of software quality and software value chain has refined the ETIS selection criteria, requiring them high level of maturity.

Teste de Software tem o objetivo de executar um programa sob teste com o objetivo de revelar suas falhas, portanto é uma das fases mais importante do ciclo de vida do desenvolvimento de um software. A testabilidade é um atributo de qualidade fundamental para o sucesso da atividade de teste, pois ela pode ser entendida como o esforço necessário para criar, executar e avaliar os casos de teste em um software. Este atributo não é uma qualidade intrínseca do software, portanto não pode ser medido diretamente como a quantidade de linhas de código, por exemplo. Entretanto, ela pode ser inferida por meio das características ou métricas internas e externas de um software. Entre as características comumente utilizadas na análise da testabilidade estão as métricas CK, que foram propostas por Chidamber e Kemerer com objetivo de analisar software orientado a objetos. A maioria dos trabalhos nesta linha, entretanto, relaciona o tamanho e a quantidade de casos testes com a testabilidade de um software. Entretanto, é fundamental analisar a qualidade dos testes para saber se eles atingem os objetivos para os quais foram propostos, independente de quantidade e tamanho. Portanto, este trabalho de mestrado apresenta um estudo empírico sobre a relação entre as métricas CK e a testabilidade de um software com base na análise da adequação de seus casos de teste unitários, critérios de teste estrutural e de mutação. Inicialmente foi realizada uma Revisão Sistemática cujo objetivo foi avaliar o estado da arte da testabilidade e as métricas CK. Os resultados mostraram que apesar de existirem várias pesquisas relacionadas ao tema, existem lacunas que motivam novas pesquisas no que concerne a análise da qualidade dos testes e identificação das características das métricas que podem ser inferidas para medir e analisar a testabilidade. Em seguida, foram realizadas duas análises empíricas. Na primeira análise, as métricas foram analisadas por meio da correlação das métricas CK com a cobertura de linha de código, cobertura de \\textit (arestas, ramos ou desvio de fluxo) e escore de mutação. Os resultados desta análise demonstraram a importância de cada métrica dentro do contexto da testabilidade. Na segunda análise, foi realizada uma proposta de clusterização das métricas para tentar identificar grupos de classes com características semelhantes relacionadas à testabilidade. Além das análises empíricas, foi desenvolvida e apresentada uma ferramenta de coleta e análise de métricas CK com objetivo de contribuir com novas pesquisas relacionados a proposta deste projeto. Apesar das limitações das análises, os resultados deste trabalho mostraram a importância de cada métrica CK dentro do contexto da testabilidade e fornece aos desenvolvedores e projetistas uma ferramenta de apoio e dados empíricos para melhor desenvolverem e projetarem seus sistemas com o objetivo de facilitar a atividade de teste de software / Software testing have aim to run a program under test with the aim of revealing its failures, so it is one of the most important phases of the software development lifecycle. Testability is a key quality attribute for the success of the test activity, because it can be understood as the effort required to create, execute and evaluate test cases in software. This attribute is not an intrinsic quality of the software, so it can not be measured directly as the number of lines code, for example. However, it can be inferred through the or internal and external metrics of a software. Among the features commonly used in testability analysis are CK metrics, which were proposed by Chidamber and Kemerer in order to analyze object-oriented software. Most of the works in this line, however, relate the size and quantity of test cases with software testability. However, it\'s critical to analyze the quality of the tests to see if they achieve the objectives for which they were proposed, independent of quantity and size. Therefore, this Master\'s degree work presents an empirical study on the relationship between CK metrics and software testability based on the analysis of the adequacy of its unit test cases, structural test criteria and mutation. Initially, a Systematic Review was carried out to evaluate the state of the art of testability and CK metrics. The results showed that although there are several researches related to the subject, there are gaps that motivate new research in what concerns the analysis of the quality of the tests and identification of the features of the metrics that can be inferred to measure and analyze the testability. Two empirical analyzes were performed. In the first analysis, the metrics were analyzed through the correlation of the CK metrics with the code line coverage, branch coverage or mutation score. The results of this analysis showed the importance of each metric within the context of testability. In the second analysis, a metric clustering proposal was made to try to identify groups of classes with similar features related to testability. In addition to the empirical analysis, a tool for the collection and analysis of CK metrics was developed and presented, with aim to contribute with new researches related to the proposal of this project. Despite the limitations of the analyzes, the results of this work showed the importance of each CK metric within the context of testability and provides developers and designers with a support tool and empirical data to better develop and design their systems with the aim of facilitate the activity of software testing

Teste de software desempenha um papel fundamental no processo de produção de um produto de software de qualidade. Com o passar dos anos, diversas técnicas e critérios de teste de software foram desenvolvidos a fim de estabelecer meios e métricas para guiar a criação de casos de teste efetivos, capazes de evidenciar defeitos no produto avaliado. Dentre os principais critérios para teste de software está o Teste de Mutação, que foi amplamente difundido e é tido como uma das abordagens mais eficazes para guiar a criação de conjuntos de casos de teste capazes de revelar defeitos em software. Entretanto, à medida que esse critério possui uma grande efetividade para revelar defeitos, ele peca pelo baixo poder de escalabilidade, o que acaba comprometendo diretamente a sua capacidade de aplicação. Neste sentido, diversos estudos foram desenvolvidos nesta área dedicando-se a aprimorar o seu desempenho e torná-lo uma alternativa viável para aplicação durante a fase de teste de software. Este trabalho apresenta indícios de que a utilização de estruturas complexas de processamento pode apoiar a aplicação do Teste de Mutação. Para tal, foi concebida uma arquitetura que possibilite a aplicação do Teste de Mutação em paralelo. Após a implementação da arquitetura foram avaliados cinco algoritmos de balanceamento de carga responsáveis por controlar a distribuição e execução do Teste de Mutação. Durante a avaliação experimental da arquitetura e dos algoritmos, observou-se que nos piores cenários avaliados foi possível atingir um ganho de desempenho acima de 70% em relação à aplicação sequencial convencional do Teste de Mutação enquanto nos melhores cenários o ganho de desempenho foi acima 95%, contudo, necessitando utilizar-se de uma infraestrutura mais robusta para a execução da arquitetura. / Software testing plays a fundamental role in the development process of quality software systems. Over the years, many software testing techniques and criteria were developed to provide means and metrics to guide the development of effective test cases, able to find defects on the product being assessed. Among major criteria for software testing is the mutation testing, which was broadly broadcast and is likely one of the most effective approaches for creating sets of test cases able to uncover software bugs. However, although mutating testing has a great effectiveness to uncover defects in a product, it suffers from low scalability, which directly compromises its applicability. In this sense, many studies were developed in this area aiming at improving the performance of that criterion and make it a viable alternative for its application throughout the software testing process. This work presents evidence that the use of complex structures of processing can support mutation testing application. For this, it was established an architecture that enables mutation testing to be performed in parallel. After implementing the architecture, five load balance algorithms to controlling the distribution and execution of mutation testing were analyzed. During the experimental evaluation of the architecture and algorithms, it was observed that in the worst evaluated scenarios it was possible to reach a gain in performance of up to 70% in comparison to the conventional application (sequential). In the best scenarios the gain was over 95% in exchange of using a more robust infrastructure for the execution of the architecture.

Um importante mecanismo da arquitetura orientada a serviços é o componente de registro de serviços. Ele permite a interação entre provedores e clientes, oferecendo um meio de acesso aos serviços desenvolvidos e publicados no registro. Nesta dissertação é proposto o desenvolvimento de um registro de serviços para dar apoio à publicação, busca e classificação de serviços Web, em particular, àqueles relacionados a ferramentas de teste de software. Uma limitação comum dos serviços de registro refere-se às buscas realizadas, pois são basicamente sintáticas e podem trazer resultados pouco relacionados aos interesses do usuário. Para resolver este problema uma ontologia de teste foi adaptada e incorporada ao registro com o objetivo de oferecer facilidades de busca e agregar informação semântica nos serviços registrados. Uma arquitetura genérica baseada em serviços para o domínio de engenharia de software é apresentada e instanciada para o domínio de teste de software com o objetivo de auxiliar no entendimento e implementação do registro de serviços proposto. Também são apresentados exemplos de ferramentas de teste publicadas no registro e um exemplo de busca e interação com o serviço de teste JaBUTiWS, previamente publicado no registro, que tem por objetivo apoiar o teste estrutural de componentes e serviços / An important mechanism of Service Oriented Architecture is the service registry (or service broker). It allows interaction among providers and consumers, offering a point to access the services developed and published in the registry. In this dissertation we propose the development of a service broker to support the publication, search and categorization of Web services, particularly those related to software testing tools. A common limitation of service brokers refers to searching facilities since they are primarily syntactic and thus can bring results that are not well related with the user\'s interest. To tackle this problem a test ontology was adapted and incorporated into the broker with the aim of improving the likelihood of finding the correct service in searches and also to add semantic information to the registered services. A generic service oriented architecture for the software engineering domain is presented and instantiated to the software testing domain with the purpose of facilitating the understanding and implementation of the proposed service registry. We also present some examples of software testing tools published in the registry and an example of search and interaction with the JaBUTiWS testing service, previously published in the registry, which aims to support structural testing of components and services