This report describes a design approach for implementing a software test automation solution that can accommodate existing test processes in an organization. The process of implementing a software test automation solution is a large undertaking and requires careful planning to avoid unsuccessful implementations. This report outlines a design that can integrate with existing business and development processes in an organization, and recommends automation and development frameworks for achieving the test automation goals. Considerations for a heterogeneous test environment with varying types of supported operating systems, such as Windows and Linux, and multiple test execution environments, such as Java and .NET, have been made in this design and in the tool selections for the system implementation. The report also describes some of the challenges and caveats of automation in a heterogeneous environment along with recommended solutions to these challenges. / text

Arquiteturas de software e teste de software desempenham um papel essencial no processo de desenvolvimento de produtos de software de alta qualidade. Com base em um mapeamento sistemático, pode-se observar que as atividades de estabelecimento de arquiteturas de software não consideram representar informações de teste de software associadas às visões da arquitetura, em particular de arquiteturas de referência. Este trabalho tem por objetivo propor um processo que induza a representação de informações de teste de software no estabelecimento de arquiteturas de referência. Dessa forma, além de se abstrair a essência de um conjunto de arquiteturas de software de um domínio de aplicação, facilitando o projeto de novas arquiteturas por meio do reúso e padronização de elementos arquiteturais, promove-se também o reúso e a padronização de informações de teste. Este trabalho é realizado a partir do modelo RAModel e do processo ProSA-RA. O RAModel apresenta um conjunto de elementos essenciais para o projeto de arquiteturas de referência. O ProSA-RA por sua vez apresenta um processo que sistematiza o estabelecimento de arquiteturas de referência, considerando os elementos do RAModel. Em particular, propõe-se um processo denominado ProSA-RAT . Esse processo viabiliza a definição de processos de teste adequados a domínios de aplicação específicos promovendo o reúso e padronização de informações de teste nas etapas iniciais de processos de desenvolvimento baseados em arquitetura. Um estudo de viabilidade do ProSA-RAT foi conduzido e um exemplo no domínio de robótica é apresentado. / Software architectures and software testing play an essential role in the development process of high quality software products. Based on a systematic mapping, it could be observed that the activities of establishing software architectures do not consider to represent software test information associated with the architecture views, in particular of reference architectures. The objective of this work is to propose a process that induces the representation of software test information in the establishment of reference architectures. Thus, in addition to abstracting the essence of a set of architectures of an application domain, facilitating the design of new architectures through the reuse and standardization of architectural elements, reuse and standardization of test information are also promoted. This work is carried out using the RAModel and the ProSA-RA. RAModel presents a set of essential elements for the design of reference architectures. ProSA-RA, on the other hand, presents a process that systematizes the establishment of reference architectures, considering the elements of the RAModel. In particular, a process named ProSA-RAT is proposed. This process facilitates the definition of test processes adequate to specific application domains by promoting the reuse and standardization of test information in the early stages of architecture based development processes. A feasibility study of ProSA-RAT has been carried out and an example in the field of robotics is presented.

Arquiteturas de software e teste de software desempenham um papel essencial no processo de desenvolvimento de produtos de software de alta qualidade. Com base em um mapeamento sistemático, pode-se observar que as atividades de estabelecimento de arquiteturas de software não consideram representar informações de teste de software associadas às visões da arquitetura, em particular de arquiteturas de referência. Este trabalho tem por objetivo propor um processo que induza a representação de informações de teste de software no estabelecimento de arquiteturas de referência. Dessa forma, além de se abstrair a essência de um conjunto de arquiteturas de software de um domínio de aplicação, facilitando o projeto de novas arquiteturas por meio do reúso e padronização de elementos arquiteturais, promove-se também o reúso e a padronização de informações de teste. Este trabalho é realizado a partir do modelo RAModel e do processo ProSA-RA. O RAModel apresenta um conjunto de elementos essenciais para o projeto de arquiteturas de referência. O ProSA-RA por sua vez apresenta um processo que sistematiza o estabelecimento de arquiteturas de referência, considerando os elementos do RAModel. Em particular, propõe-se um processo denominado ProSA-RAT . Esse processo viabiliza a definição de processos de teste adequados a domínios de aplicação específicos promovendo o reúso e padronização de informações de teste nas etapas iniciais de processos de desenvolvimento baseados em arquitetura. Um estudo de viabilidade do ProSA-RAT foi conduzido e um exemplo no domínio de robótica é apresentado. / Software architectures and software testing play an essential role in the development process of high quality software products. Based on a systematic mapping, it could be observed that the activities of establishing software architectures do not consider to represent software test information associated with the architecture views, in particular of reference architectures. The objective of this work is to propose a process that induces the representation of software test information in the establishment of reference architectures. Thus, in addition to abstracting the essence of a set of architectures of an application domain, facilitating the design of new architectures through the reuse and standardization of architectural elements, reuse and standardization of test information are also promoted. This work is carried out using the RAModel and the ProSA-RA. RAModel presents a set of essential elements for the design of reference architectures. ProSA-RA, on the other hand, presents a process that systematizes the establishment of reference architectures, considering the elements of the RAModel. In particular, a process named ProSA-RAT is proposed. This process facilitates the definition of test processes adequate to specific application domains by promoting the reuse and standardization of test information in the early stages of architecture based development processes. A feasibility study of ProSA-RAT has been carried out and an example in the field of robotics is presented.

Software metrics plays an important role in measuring attributes that are critical to the success of a software project. Measurement of these attributes helps to make the characteristics and relationships between the attributes clearer. This in turn supports informed decision making. The field of software engineering is affected by infrequent, incomplete and inconsistent measurements. Software testing is an integral part of software development, providing opportunities for measurement of process attributes. The measurement of software testing process attributes enables the management to have better insight in to the software testing process. The aim of this thesis is to investigate the metric support for software test planning and test design processes. The study comprises of an extensive literature study and follows a methodical approach. This approach consists of two steps. The first step comprises of analyzing key phases in software testing life cycle, inputs required for starting the software test planning and design processes and metrics indicating the end of software test planning and test design processes. After establishing a basic understanding of the related concepts, the second step identifies the attributes of software test planning and test design processes including metric support for each of the identified attributes. The results of the literature survey showed that there are a number of different measurable attributes for software test planning and test design processes. The study partitioned these attributes in multiple categories for software test planning and test design processes. For each of these attributes, different existing measurements are studied. A consolidation of these measurements is presented in this thesis which is intended to provide an opportunity for management to consider improvement in these processes. / 00 92 51 4430327

Made available in DSpace on 2015-04-11T14:03:11Z (GMT). No. of bitstreams: 1 DISSERTACAO DANIELLA.pdf: 2040757 bytes, checksum: 21b1bef3ce0330932763dc1d246abb39 (MD5) Previous issue date: 2008-02-22 / Fundação de Amparo à Pesquisa do Estado do Amazonas / This work encompasses elements of a study of knowlwedge representation founded on ontologies that have Linux testing as target domain. The study aims at demonstrating that once knowledge is formalised, it is possible to reuse it, to perform inference, to process it throuch computers, and, what is more, it becames amenable to being communicated between people and software. Towards that, three ontlogies have been developed: OSOnto (Operating System Ontology) which represents concepts of the operating system domain, SwTO (Software Test Ontology) which deals with the software testing domain, and SwTOI (SwTO Integrated) which represents concepts of both the above domains in an integrated way. For implementing the ontologies, OWL DL as ontology specification language, Prot´eg´e as ontology edition environment and Racer as the main reasoner, have been used. A quantitative and qualitative evaluation of the SwTOI ontology has been performed. / Este trabalho reúne elementos de um estudo sobre representacão do conhecimento fundamentado em ontologias tendo como domínio alvo o teste do Linux. O estudo visa demonstrar que uma vez que o conhecimento é formalizado, é possível reusá-lo, realizar inferência, processá-lo computacionalmente, como também tornase passível de comunicação entre pessoas e software. Para tal, foram desenvolvidas três ontologias: a OSOnto (Operating System Ontology) que representa conceitos do domínio de Sistema Operacional, a SwTO (Software Test Ontology) que trata do dom´ıniodetestedesoftware, e a SwTOI (SwTOIntegrated) que representa conceitos destes dois domínios integrados. Para a implementação das ontologias foi utilizada OWL DL como linguagem de especificação, o Protégé como ambiente de edição e o Racer como principal raciocinador. Uma avaliação quantitativa e qualitativa foi realizada da ontologia SwTOI.

Software testing is a tedious and very expensive undertaking. Automatic test data generation is, therefore, proposed in this research to help testers reduce their work as well as ascertain software quality. The concept of test driven development (TDD) has become increasingly popular during the past several years. According to TDD, test data should be prepared before the beginning of code implementation. Therefore, this research asserts that the test data should be generated from the software design documents which are normally created prior to software code implementation. Among such design documents, the UML state machine diagrams are selected as a platform for the proposed automated test data generation mechanism. Such diagrams are selected because they show behaviours of a single object in the system. The genetic algorithm (GA) based approach has been developed and applied in the process of searching for the right amount of quality test data. Finally, the generated test data have been used together with UML class diagrams for JUnit test code generation. The GA-based test data generation methods have been enhanced to take care of parallel path and loop problems of the UML state machines. In addition the proposed GA-based approach is also targeted to solve the diagrams with parameterised triggers. As a result, the proposed framework generates test data from the basic state machine diagram and the basic class diagram without any additional nonstandard information, while most other approaches require additional information or the generation of test data from other formal languages. The transition coverage values for the introduced approach here are also high; therefore, the generated test data can cover most of the behaviour of the system. / EU Asia-Link project TH/Asia Link/004(91712) East-West and CAMT

Software testing is a tedious and very expensive undertaking. Automatic test data generation is, therefore, proposed in this research to help testers reduce their work as well as ascertain software quality. The concept of test driven development (TDD) has become increasingly popular during the past several years. According to TDD, test data should be prepared before the beginning of code implementation. Therefore, this research asserts that the test data should be generated from the software design documents which are normally created prior to software code implementation. Among such design documents, the UML state machine diagrams are selected as a platform for the proposed automated test data generation mechanism. Such diagrams are selected because they show behaviours of a single object in the system. The genetic algorithm (GA) based approach has been developed and applied in the process of searching for the right amount of quality test data. Finally, the generated test data have been used together with UML class diagrams for JUnit test code generation. The GA-based test data generation methods have been enhanced to take care of parallel path and loop problems of the UML state machines. In addition the proposed GA-based approach is also targeted to solve the diagrams with parameterised triggers. As a result, the proposed framework generates test data from the basic state machine diagram and the basic class diagram without any additional nonstandard information, while most other approaches require additional information or the generation of test data from other formal languages. The transition coverage values for the introduced approach here are also high; therefore, the generated test data can cover most of the behaviour of the system.

Mobile applications are developed to run on fast-evolving platforms, such as Android or iOS. Respective mobile devices are heterogeneous concerning hardware (e.g., sensors, displays, communication interfaces) and software, especially operating system functions. Software vendors cope with platform evolution and various hardware configurations by abstracting from these variable assets. However, they cannot be sure about their assumptions on the inner conformance of all device parts and that the application runs reliably on each of them—in consequence, comprehensive testing is required. Thereby, in testing, variability becomes tedious due to the large number of test cases required to validate behavior on all possible device configurations. In this paper, we provide remedy to this problem by combining model-based testing with variability concepts from Software Product Line engineering. For this purpose, we use feature-based test modeling to generate test cases from variable operational models for individual application configurations and versions. Furthermore, we illustrate our concepts using the commercial mobile application “runtastic” as example application.

Arquitetura de software e teste de software são importantes áreas com o objetivo comum de prover meios para que sistemas de software sejam desenvolvidos com alta qualidade. Foi constatado, por meio de um mapeamento sistemático, que existem poucos estudos relacionando essas duas áreas de modo que uma possa contribuir para a outra, principalmente em arquiteturas de referência e teste de software. Esse projeto tem por objetivo estender o modelo para arquiteturas de referência (RAModel), para que arquiteturas de referência sejam estabelecidas com informações de teste de software. Assim, além de promover padronização, uniformização e reúso a informações relacionadas a arquiteturas de software, promove-se também o reúso, a padronização e a uniformização das informações de teste de software no desenvolvimento das soluções para um dado domínio de aplicação. Para isso, elementos de teste relacionados ao planejamento de teste foram extraídos do TMMi e da norma ISO/IEC/IEEE 29119 e inseridos no RAModel, definindo, assim, o RAModelT . Dessa forma, a atividade de teste é incorporada aos estágios iniciais do desenvolvimento de software. Um estudo de viabilidade realizado para avaliar a extensão do RAModelT e um exemplo de aplicação são apresentados. / Software architecture and software testing are important areas with the common goal of providing the means to develop high-quality software systems. It was found through a systematic mapping that there are few studies relating these two areas so that one can contribute to the other, especially in reference architectures and software testing. This project aims at extending the Reference Architecture Model (RAModel), so that reference architectures are established with software test information. Thus, in addition to promoting standardization, standardization, and reuse of information related to software architectures, it is also promoted the reuse, standardization, and standardization of software test information in the development of solutions for a given application domain. For this, test elements related to the test planning were extracted from the TMMi and the ISO/IEC/IEEE 29119 standard and inserted into the RAModel, thus defining the RAModelT . In this way, the test activity is incorporated into the early stages of software development. A feasibility study performed to evaluate the extent of the RAModelT and an application example are presented.

Testing is the dominating method for quality assurance of industrial software. Despite its importance and the vast amount of resources invested, there are surprisingly limited efforts spent on testing research, and the few industrially applicable results that emerge are rarely adopted by industry. At the same time, the software industry is in dire need of better support for testing its software within the limited time available. Our aim is to provide a better understanding of how test cases are created and applied, and what factors really impact the quality of the actual test. The plethora of test design techniques (TDTs) available makes decisions on how to test a difficult choice. Which techniques should be chosen and where in the software should they be applied? Are there any particular benefits of using a specific TDT? Which techniques are effective? Which can you automate? What is the most beneficial way to do a systematic test of a system? This thesis attempts to answer some of these questions by providing a set of guidelines for test design, including concrete suggestions for how to improve testing of industrial software systems, thereby contributing to an improved overall system quality. The guidelines are based on ten studies on the understanding and use of TDTs. The studies have been performed in a variety of system domains and consider several different aspects of software test. For example, we have investigated some of the common mistakes in creating test cases that can lead to poor and costly testing. We have also compared the effectiveness of different TDTs for different types of systems. One of the key factors for these comparisons is a profound understanding of faults and their propagation in different systems. Furthermore, we introduce a taxonomy for TDTs based on their effectiveness (fault finding ability), efficiency (fault finding rate), and applicability. Our goal is to provide an improved basis for making well-founded decisions regarding software testing, together with a better understanding of the complex process of test design and test case writing. Our guidelines are expected to lead to improvements in testing of complex industrial software, as well as to higher product quality and shorter time to market.