A qualidade dos sistemas de software é uma preocupação de todo bom projeto e muito tem se estudado para melhorar tanto a qualidade do produto final quanto do processo de desenvolvimento. Teste de Software é uma área de estudo que tem crescido significativamente nos últimos tempos, em especial a automação de testes que está cada vez mais em evidência devido à agilidade e qualidade que pode trazer para o desenvolvimento de sistemas de software. Os testes automatizados podem ser eficazes e de baixo custo de implementação e manutenção e funcionam como um bom mecanismo para controlar a qualidade de sistemas. No entanto, pouco conhecimento sobre a área e erros comuns na escrita e manutenção dos testes podem trazer dificuldades adicionais aos projetos de software. Testes automatizados de baixa qualidade não contribuem efetivamente com o controle de qualidade dos sistemas e ainda demandam muito tempo do desenvolvimento. Para evitar esses problemas, esta dissertação apresenta de forma crítica e sistemática as principais práticas, padrões e técnicas para guiar o processo da criação, manutenção e gerenciamento dos casos de testes automatizados. Inicialmente, são feitas comparações entre a automação de testes e outras práticas de controle e garantia de qualidade. Em seguida, são apresentados os problemas e soluções mais comuns durante a automação de testes, tais como questões relacionadas a tipos específicos de algoritmos, sistemas com persistência de dados, testes de interfaces de usuário e técnicas de desenvolvimento de software com testes automatizados. Para finalizar, a dissertação traz uma reflexão sobre o gerenciamento e a abordagem da automação de testes para tornar o processo mais produtivo e eficaz. / The quality of software systems is a concern of every good project and much has been studied to improve the quality of the final product and process development. Software Testing is an increasing area, especially test automation, which is in evidence due to the speed and quality that it may bring to the development of software systems. Automated tests can be effective and can have a low cost of implementation and maintenance to ensure and control the quality of the systems. However, little knowledge about the area and common errors in writing and maintaining tests may bring additional difficulties to the software projects. Low quality automated tests do not contribute effectively to quality control systems and still take a long time of development. To avoid these problems, we present critically and systematically the core practices, standards and techniques to guide the process of creation, maintenance and management of automated test cases. Initially, comparisons are made between the test automation, other control practices, and quality assurance. Next, we present the most common problems and solutions for the automation of tests, such as issues related to specific types of algorithms, systems with data persistence, testing user interfaces and techniques for software development with automated tests. Finally, this essay reflects on the management and approach to test automation to make the process more productive and effective.

Submitted by Geyciane Santos (geyciane_thamires@hotmail.com) on 2015-06-17T13:05:26Z No. of bitstreams: 1 Disssertação- Jardelane de Brito Costa.pdf: 2340050 bytes, checksum: 5f23cc3aa127017a2229e21ce654aba5 (MD5) / Approved for entry into archive by Divisão de Documentação/BC Biblioteca Central (ddbc@ufam.edu.br) on 2015-06-17T20:24:38Z (GMT) No. of bitstreams: 1 Disssertação- Jardelane de Brito Costa.pdf: 2340050 bytes, checksum: 5f23cc3aa127017a2229e21ce654aba5 (MD5) / Made available in DSpace on 2015-06-17T20:24:38Z (GMT). No. of bitstreams: 1 Disssertação- Jardelane de Brito Costa.pdf: 2340050 bytes, checksum: 5f23cc3aa127017a2229e21ce654aba5 (MD5) Previous issue date: 2012-04-30 / Outras / The quality of testing applied to a software project is an important factor to reach the quality of the final product. This becomes more evident in methodologies based on testing to measure the software development progress or that consider testing the central element in the evaluation of its quality. Several strategies can be applied to evaluate the quality of testing in a software project. A possible strategy is to ensure the quality of artifacts produced during testing process. In this context, a technique that has been largely used to ensure the quality of software artifacts is inspection. There are many works dealing with inspection of documents produced during the software development process. However, we have not identified in the technical literature works that apply inspection in software testing artifacts. We identified just studies indicating the importance of applying inspection in these artifacts. Thus, with the purpose of evaluating the quality of testing artifacts, contributing to defects detection before tests execution, this thesis presents a checklist-based approach, TestCheck, for inspection of software testing artifacts (test plans, cases, and procedures). In order to evaluate the proposed approach, experimental studies (feasibility, observation and case studies) were conducted. They analyzed and indicated the efficacy, efficiency, and maturity of the proposed approach in relation to detection of defects in testing artifacts. / A qualidade dos testes aplicados em um projeto de Software é um fator determinante para a qualidade do produto final. Isso se torna mais evidente em metodologias que se baseiam em testes para progressão do desenvolvimento do Software ou consideram os testes pontos centrais da avaliação da sua qualidade. Diversas estratégias podem ser aplicadas para avaliação da qualidade dos testes em um projeto de Software. Uma delas é garantir a qualidade dos artefatos de testes produzidos ao longo do processo de testes. Neste contexto, uma técnica que vem sendo bastante utilizada para garantir a qualidade de artefatos de Software é inspeção. Existem diversos trabalhos que abordam inspeção em documentos produzidos ao longo do processo de desenvolvimento de software. No entanto, não foram encontrados na literatura técnica trabalhos que utilize inspeção em artefatos de Teste de Software. O que existe são trabalhos que mencionam a importância de se fazer inspeção nesses artefatos. Assim, com o objetivo de avaliar a qualidade dos artefatos de teste, proporcionando que defeitos sejam encontrados antes da execução dos testes, esta dissertação apresenta uma abordagem baseada em Checklist, TestCheck, para inspeção de artefatos de teste de Software (planos, casos e procedimentos de teste). Para avaliar a abordagem proposta, foram realizados estudos experimentais (estudo de viabilidade, segundo estudo de viabilidade e estudo de observação na indústria). Eles analisaram e indicaram a eficácia, eficiência e amadurecimento da abordagem proposta em relação à detecção de defeitos em artefatos de teste.

O teste de programas concorrentes e uma atividade desaadora, devido a fatores que não estão presentes em programas sequenciais, como comunicação, sincronização e não determinismo. Algumas técnicas de teste têm sido propostas para o contexto de programação concorrente, mas raramente sua aplicabilidade e avaliada por estudos teóricos ou experimentais. Este trabalho contribui nesse sentido, propondo e conduzindo um estudo experimental para avaliar o custo, eficácia e aspecto complementar dos critérios de teste estruturais para programas concorrentes no contexto de memória compartilhada, implementados usando o padrão PThreads (Posix Threads). A ferramenta de teste ValiPThread e usada para auxiliar a condução do experimento. Os programas usados no experimento foram selecionados de benchmarks, como o Inspect, Helgrind e Rungta. Esses benchmarks são comumente usados no estudo de técnicas de teste para programas concorrentes. Programas que resolvem problemas clássicos da programação concorrente também foram incluídos no estudo. Com base nos resultados obtidos foi definida uma estratégia de aplicação, considerando aspectos de custo e eficácia dos critérios de teste. Além disso, todo o material utilizado e gerado durante o experimento foi reunido em um pacote de laboratório, a fim de contribuir com a comunidade de pesquisa, possibilitando replicações e comparações desses critérios com outras técnicas de teste no contexto de programas concorrentes / Concurrent program testing is a challenging activity due to the communication, synchronization and nondeterminism of this application domain. Despite that, some testing techniques for concurrent programs have been proposed, but their applicability is rarely evaluated by theoretical or experimental studies. This work contributes in this direction proposing and conducting an experimental study to evaluate the cost, effectiveness and strength of structural testing criteria for multithreaded programs, implemented using the Pthreads standard (POSIX Threads). The testing tool ValiPThread is used to support the conduction of the experiment. The programs used in this experiment were selected from classical benchmarks, such as Inspect, Helgring and Rungta. These benchmarks are commonly used to study testing techniques for concurrent programs. We also include programs that solve concurrent classical problems. Based on the obtained results, we defined an application testing strategy, considering cost and effectiveness aspects of the testing criteria. Furthermore, all material used and generated during the experiment was incorporated in a lab package, in order to contribute with further research studies making possible replications and comparisons of these testing criteria with other testing techniques in context of concurrent programs

Thesis (DPhil (Informatics))--Cape Peninsula University of Technology, 2018. / Increasingly, organisations in South African and across the world rely on software for various reasons, such as competitiveness and sustainability. The software are either developed in-house or purchased from the shelf. Irrespective of how the software was acquired, they do encounter challenges, from implementation to support, and use stages. The challenges sometimes hinder and are prohibitive to processes and activities that the software is intended to enable and support. Majority of the challenges that are encountered with software are attributed to the fact that they were not tested or appropriately tested before implementation. Some of the challenges has been costly to many organisations, particularly in South Africa. As a result, some organisations have been lacking in their efforts toward growth, competitiveness and sustainability. The challenges manifest from the fact that there are no testing tools and methods that can be easily customised for an organisation’s purposes. As a result, some organisations adopt more tools and methods for the same testing purposes, which has not solved the problem, as the challenges continue among South Africa organisations. Based on the challenges as stated above, this study was undertaken. The aim was to develop a decision support system framework, which can be used for software testing by any organisation, owing to its flexibility for customisation. The interpretivist and inductive approaches were employed. The qualitative methods and the case study design approach were applied. Three South African organisations, a private, public and small to medium enterprise (SME) were used as cases in this study. A set of criteria was used to select the organisations. The analysis of the data was guided by two sociotechnical theories, actor network theory (ANT) and diffusion of innovation (DOI). The theories were complementarily applied because of their different focuses. The actor network theory focuses on actors, which are both human and non-human, heterogeneity of networks, and the relationship between the actors within networks. This includes the interactions that happen at different moments as translated within the heterogeneous networks. Thus, ANT was employed to examine and gain better understanding of the factors that influence software testing in organisations. The DOI focuses on how new (fresh) ideas are diffused in an environment, with particular focus on innovation decision process, which constitute five stages: knowledge, persuasion, decision, implementation and confirmation. Findings from the data analysis of the three cases were further interpreted. Based on the interpretation, a decision support system framework was developed. The framework is intended to be of interest to software developers, software project managers and other stakeholders, most importantly, to provide guide to software testers in their tasks of testing software. Thus, this research is intended to be of interest and benefit to organisations and academic through its theoretical, practical and methodological contribution as detailed in the chapter seven (conclusion). In conclusion, even though this research is rigorous, comprehensive and holistic, there are room for future studies. I would like to propose that future research should be in the areas of measurement of software testing. Also, sociotechnical theories like structuration theory and technology acceptance model should be considered in the analysis of such studies.

O teste baseado em modelos tem como objetivo auxiliar a atividade de testes, gerando conjuntos de casos de teste a partir de modelos, como Máquinas de Estados Finitos (MEFs). Diversos métodos de geração de conjuntos de caso de teste têm sido propostos ao longo das últimas décadas, com algumas contribuições recentes. Dentre esses trabalhos, há os que geram seqüências de verificação que são conjuntos de caso de teste formados por uma única seqüência e que são capazes de detectar os defeitos de uma implementação cujo comportamento pode ser modelado a partir de uma MEF. Neste trabalho é proposto um algoritmo de geração de seqüências de verificação que tem a finalidade de gerar seqüências menores que as seqüências geradas pelos métodos existentes. O algoritmo, que é baseado na técnica de algoritmos genéticos e nas condições de suficiência para a completude de casos de teste, consiste basicamente em criar novas seqüências a partir de seqüências menores. Por meio de mutações, novas seqüências são geradas pelo algoritmo. As condições de suficiência são utilizadas para determinar quais seqüências geradas são seqüências de verificação. Também são apresentados neste trabalho os estudos experimentais realizados para determinar o comportamento do algoritmo diante de diferentes contextos / Model-based testing aims at aiding the testing activity, generating test cases from models such as Finite State Machines (FSM). Several test cases generation methods have been proposed along the last decades, with some recent contributions. Among these works, there are those that generate checking sequences, which are test cases formed by a single sequence and which are capable of detecting faults in an implementation whose behavior can be modeled as an FSM. This work proposes a checking sequences generation algorithm which aims at generating sequences smaller than the sequences generated by existing methods. The algorithm, which is based on the genetic algorithms technique and sufficient conditions for completeness of test cases, basically consists of creating new sequences from small sequences. Through mutations, new sequences are generated by the algorithm. The suffcient conditions are used to determine which sequences are checking sequences. Experimental studies are presented in this work to determine the behavior of the algorithm on different contexts

O teste de software é reconhecido como uma importante atividade na garantia da qualidade de sistemas de software. Com o objetivo de dar apoio a essa atividade, uma diversidade de ferramentas de teste têm sido desenvolvida. Entretanto, grande parte dessas ferramentas é construída de forma isolada, possuindo arquiteturas e estruturas próprias, o que tem impactado negativamente a capacidade de integração e o reúso dessas ferramentas. Nesse contexto, esforços têm sido dedicados à disponibilização de ferramentas de teste orientadas a serviço, ou seja, ferramentas que são baseadas na SOA (Service Oriented Architecture). Em uma outra perspectiva, arquiteturas de referência têm desempenhado um importante papel no desenvolvimento de sistemas de software, uma vez que contém informações sobre como desenvolver sistemas para um determinado domínio de aplicação, buscando contribuir para o sucesso de sistemas desse domínio. Assim, o principal objetivo deste trabalho é o estabelecimento de uma arquitetura de referência orientada a serviço, denominada RefTEST-SOA (Reference Architecture for Software Testing Tools based on SOA), que agrega o conhecimento e a experiência de como organizar ferramentas de teste orientadas a serviço, visando também à integração, à escalabilidade e o reúso providos pela SOA. Para o estabelecimento dessa arquitetura, foi utilizado o ProSA-RA, um processo que sistematiza o projeto, representação e avaliação de arquiteturas de referência. Resultados alcançados no estudo de caso conduzido evidenciam que a RefTEST-SOA é uma arquitetura viável e reusável para o desenvolvimento de ferramentas de teste orientadas a serviço / Software testing is considered as an important activity to ensure the quality of software systems. To support such activity, a diversity of testing tools have been developed. However, most of them have been separately built and have usually their particular structures and architectures, which has hindered the integration and reuse of these tools. In this context, efforts have been employed in order to provide service-oriented testing tools, i.e., tools that are based on SOA (Service Oriented Architecture). In another perspective, reference architectures have played an important role in the development of software systems, since they contain information about how to develop systems for a particular application domain, contributing to the success of systems in that domain. Thereby, our main objective is to establish a service-oriented reference architecture, named RefTEST-SOA (Reference Architecture for Software Testing Tools based on SOA), which aggregates the knowledge and experience about how to organize service-oriented testing tools, also aiming at integration, scalability and reuse provided by SOA. To establish this architecture, we have used ProSA-RA, a process that provides guidelines to the design, representation and evaluation of reference architectures. Results achieved by a conducted case study indicate that RefTEST-SOA is a viable and reusable architecture for developing service-oriented testing tools

Contexto: Apesar de existirem muitas informações sobre a área de teste de software, elas se encontram de forma dispersa e sem conexão, o que aumenta a já existente dificuldade por parte de usuários em compreender os conceitos e as tecnologias dessa área e, conseqüentemente, em tomar a decisão de onde e quando usálas. Objetivo: O objetivo deste trabalho foi criar o arcabouço KITest (Knowledge and Improvement on Test) capaz de agregar o conhecimento em teste e disponibilizá-lo para a comunidade com a intenção de facilitar a sua transferência, a definição e a melhoria de processos de teste, com mais qualidade. Metodologia: Para facilitar a transferência de conhecimento modelou-se o conhecimento em teste por meio de um processo genérico de teste organizado em um mapa mental (KITMap). Para contemplar a questão de qualidade estabeleceu-se como base as práticas do modelo TMMi, distribuídas no processo de teste genérico. Para permitir que a comunidade interaja com essa base de conhecimento em teste criou-se uma ferramenta (KITTool) que permita acesso a essas informações, faça diagnóstico do processo de teste vigente e sugira melhorias. Para gerenciar toda essa estrutura utilizou-se uma estratégia de melhoria para que essa estrutura esteja sempre em evolução com base na realimentação da comunidade que a utiliza. Resultados e Conclusões: os resultados e as conclusões sobre a aplicação do arcabouço KITest estão apresentados no relatório técnico anexo a esta tese / Background: Although there is much information about the software testing area, they are dispersed and disconnected, thus hardening the understanding of concepts and technologies within this area. Consequently, this increases the difficulty on making decisions on where and when to use such testing-related information. Objective: The objective of this work was to create the KITest framework (Knowledge and Improvement on Test) aiming at aggregating knowledge of software testing and making it available to the community. The purpose of KITest is facilitating the knowledge transfer and supporting the definition and improvement of testing processes with respect to its quality. Methodology: Aiming to facilitate the knowledge transfer, the testingrelated knowledge was modeled through a generic testing process organized as a mental map (KITMap). TMMi was chosen as the underlying model to address the intended quality issues. The TMMi practices were distributed into the general testing process. To allow the communitys interaction with the testing knowledge base, we created a tool named KITTool. The tool allows the access to the testing-related information, supports the diagnosis of the current testing process and the suggestion of improvements to it. To manage all this structure, an improvement strategy is used to maintain a continuous evolution based on the communitys feedback. Results and Conclusions: The results and conclusions on the implementation of the KITest framework are described in the technical report that is attached to this dissertation

The thesis work describes the development of a multiagent testing application (MTA) based on an agent approach for solving challenges in regression testing domain, such as: reducing the complexity of testing, optimizing the time consumption, increasing the efficiency and implementing the automation of this approach for regression testing. All these challenges related to effectiveness and cost, can be represented as measures of achieved code coverage and number of test cases created. A multiagent approach is proposed in this thesis since it allows the implementation of the autonomous behaviour and optimizes the data processing in a heterogeneous environment. In addition, the agent-based approach provides flexible design methods for building multitask applications and conducting parallel task execution. However, all of these advantages of using an agent-based approach need to be investigated in the regression testing domain for realistic scenarios. Therefore, a hypothesis was formulated in order to investigate the efficiency of the MTA approach using an experiment as the main research method for obtaining results. The thesis includes a comparison analysis between the MTA and well-known test case generation tools (i.e. EvoSuite and JUnitTools) for identifying the differences in terms of efficiency and code coverage achieved. The comparison results showed advantages of the MTA within regression testing context due to optimal level of code coverage and test cases. The outcome of the thesis work moves toward solving the aforementioned problems in regression testing domain and shows some advantages of using the multagent approach within regression testing context.

No description available.

<p>Model Based Testing (MBT) is a new-age test automation technique traditionally used for Functional Black-Box Testing. Its capability of generating test cases by using model developed from the analysis of the abstract behavior of the System under Test is gaining popularity. Many commercial and open source MBT tools are available currently in market. But each one has its own specific way of modeling and test case generation mechanism that is suitable for varied types of systems. Ericsson, a telecommunication equipment provider company, is currently adapting Model Based Testing in some of its divisions for functional testing. Those divisions haven’t yet attempted adapting Model Based Testing for non-functional testing in a full-pledged manner. A comparative study between various MBT tools will help one of the Ericsson’s testing divisions to select the best tool for adapting to its existing test environment. This also helps in improving the quality of testing while reducing cost, time and effort. This thesis work helps Ericsson testing division to select such an effective MBT tool. Based on aspects such as functionality, flexibility, adaptability, performance etc., a comparative study is carried out on various available MBT tools and a few were selected among them: Qtronic, ModelJUnit and Elvior Motes.This thesis also helps to understand the usability of the selected tools for modeling of non-functional requirements using a new method. A brief idea of modeling the non-functional requirements is suggested in this thesis. A System under Test was identified and its functional behavior was modeled along with the non functional requirements in Qtronic and ModelJUnit. An experimental analysis, backed by observations of using the new proposed method indicates that the method is efficient enough to carry out modeling non-functional requirements along with modeling of functional requirements by identifying the appropriate approach.Model Based Testing (MBT) is a new-age test automation technique traditionally used for Functional Black-Box Testing. Its capability of generating test cases by using model developed from the analysis of the abstract behavior of the System under Test is gaining popularity. Many commercial and open source MBT tools are available currently in market. But each one has its own specific way of modeling and test case generation mechanism that is suitable for varied types of systems. Ericsson, a telecommunication equipment provider company, is currently adapting Model Based Testing in some of its divisions for functional testing. Those divisions haven’t yet attempted adapting Model Based Testing for non-functional testing in a full-pledged manner. A comparative study between various MBT tools will help one of the Ericsson’s testing divisions to select the best tool for adapting to its existing test environment. This also helps in improving the quality of testing while reducing cost, time and effort. This thesis work helps Ericsson testing division to select such an effective MBT tool. Based on aspects such as functionality, flexibility, adaptability, performance etc., a comparative study is carried out on various available MBT tools and a few were selected among them: Qtronic, ModelJUnit and Elvior Motes.</p><p>This thesis also helps to understand the usability of the selected tools for modeling of non-functional requirements using a new method. A brief idea of modeling the non-functional requirements is suggested in this thesis. A System under Test was identified and its functional behavior was modeled along with the non functional requirements in Qtronic and ModelJUnit. An experimental analysis, backed by observations of using the new proposed method indicates that the method is efficient enough to carry out modeling non-functional requirements along with modeling of functional requirements by identifying the appropriate approach.</p>