Tests are important artifacts in the software development process. Testing activities such as test automation, test maintenance, and test suite optimization mainly rely on an in-depth understanding of test specifications. The manual process of writing testspecifications in natural language can create many different quality issues such as ambiguous, incomplete, redundant, or inconsistent test cases. Nowadays, the concept of test smells is proposed by several researchers to be used as indicators of low-qualityattributes in test specifications. Quality assurance processes for test specifications often rely on manual reviews to detect these smells. The manual process of detecting these smells is considered time consuming and costly. However, there is currently no work that implements a comprehensive quality model that classifies and identifies these smells by using a systematic strategy. As a result, there is a need for machine-supported analytical measures that decrease the time and effort needed to detect these smells manually, especially when it comes to reviewing and validating large test specifications.This study aims to investigate which natural language smell metrics implemented in NALABS can be found in test specifications and to measure the sufficiency of those smellmetrics. It also aims to extend these smell metrics by exploring, proposing, or combining with new bad smell metrics to cover more aspects of natural language test quality. The results of the study show that the smell metrics exists in real-world test specifications and can uncover many potential quality issues by assisting test designers in identifying certain types of quality issues pertaining to for example the understandability and complexity of test specifications. Moreover, the results show thatthe list of smell metrics implemented in NALABS is incomplete and can be extended to cover more aspects of test quality.

Many real-time systems are developed and maintained through the use of commercial software products, such as Matlab and MatrixX, that automatically generate source code based on graphical control systems models. Testing these real-time models and the real-time software generated from them presents special problems during maintenance not faced with other forms of software. Very importantly, many of the models and software systems have to be tested through the use of simulations. Huge input and output data sets, the need for testing over a long duration of time (weeks or months), and computationally intensive requirements are just a few of the difficulties. For testing during maintenance in such situations, this paper draws upon field experience to present a set of test types and a strategy for selecting test types used to create series of input values to serve as test cases. Also this paper presents strategies for applying these test types, using the assistance of a free, widely available testing tool that automates test case generation, executes the simulations, and supports the analysis of the test results.

As software systems have grown in size and complexity, the costs of maintaining such systems increases steadily. In the early 2000's, IBM launched the autonomic computing initiative to mitigate this problem by injecting feedback control mechanisms into software systems to enable them to observe their health and self-heal without human intervention and thereby cope with certain changes in their requirements and environments. Self-healing is one of several fundamental challenges addressed and includes software systems that are able to recover from failure conditions. There has been considerable research on software architectures with feedback loops that allow a multi-component system to adjust certain parameters automatically in response to changes in its environment. However, modifying the components' source code in response to failures remains an open and formidable challenge. Automatic program repair techniques aim to create and apply source code patches autonomously. These techniques have evolved over the years to take advantage of advancements in programming languages, such as reflection. However, these techniques require mechanisms to evaluate if a candidate patch solves the failure condition. Some rely on test cases that capture the context under which the program failed---the patch applied can then be considered as a successful patch if the test result changes from failing to passing. Although test cases are an effective mechanism to govern the applicability of potential patches, the automatic generation of test cases for a given scenario has not received much attention. ReCrash represents the only known implementation to generate test cases automatically with promising results through the use of low-level instrumentation libraries. The work reported in this thesis aims to explore this area further and under a different light. It proposes the use of Aspect-Oriented Programming (AOP)---and in particular of AspectJ---as a higher-level paradigm to express the code elements on which monitoring actions can be interleaved with the source code, to create a representation of the context at the most relevant moments of the execution, so that if the code fails, the contextual representation is retained and used at a later time to automatically write a test case. By doing this, the author intends to contribute to fill the gap that prevents the use of automatic program repair techniques in a self-healing architecture. The prototype implementation engineered as part of this research was evaluated along three dimensions: memory usage, execution time and binary size. The evaluation results suggest that (1) AspectJ introduces significant overhead with respect to execution time, (2) the implementation algorithm causes a tremendous strain on garbage collection, and (3) AspectJ incorporates tens of additional lines of code, which account for a mean size increase to every binary file of a factor of ten compared to the original size. The comparative analysis with ReCrash shows that the algorithm and data structures developed in this thesis produce more thorough test cases than ReCrash. Most notably, the solution presented here mitigates ReCrash's current inability to reproduce environment-specific failure conditions derived from on-demand instantiation. This work can potentially be extended to apply in less-intrusive frameworks that operate at the same level as AOP to address the shortcomings identified in this analysis. / Graduate

A lot of research is done in requirements engineering and testing but often the extensive literature is missing on defining good methods for linking functional requirements with test cases. Most of the delays occurring in the software development projects are because of incomplete or inaccurate functional requirements. The two main goals of our project are to achieve a successful software project by First, to design a template, which will merge functional requirements with test cases and second is to find the benefits of the aligning requirements to test cases. Changing, updating and tracing the requirements during the development of the project is not an easy task. The main reason for project failure is due to possibility of not fulfilling specified project requirements, so one way to solve this problem is to merge functional requirements with test cases. Thus removes the need of creating a separate requirements document, which will improve the traceability process between requirements and testing, thus leads to high quality and efficient development. The template helps us to drive a successful project by identifying the issues at an earlier stage of the development cycle.

Šiame dokumente aprašytas darbas susideda iš trijų pagrindinių dalių. Pirmojoje (analizės) dalyje yra apžvelgiamos programinės įrangos kokybės užtikrinimo veiklos – konkrečiai testavimo procesas. Didžiausias dėmesys yra skiriamas automatizuotam testų generavimui. Antrojoje (projektinėje) dalyje aprašomas simboliniu vykdymu grindžiamas mutacinis testų generavimo metodas. Taip pat detalizuojamas jo realizavimas kuriamoje sistemoje – aprašomi statiniai ir dinaminiai vaizdai. Trečiojoje dalyje (tyrimo ir eksperimentinėje) yra analizuojamas sukurtas metodas, vertinamos įvairiausios jo charakteristikos, metrikos ir realizuojami patobulinimai. Šie sistemos priežiūros darbai leido sumažinti ciklomatinį metodų sudėtingumą ir pagreitinti realizuoto testų generavimo metodo veikimą. Aprašytas metodas pasižymi tuo, jog jo pagalba galima sugeneruoti testus aptinkančius programinio kodo mutacijas, o testų generavimui yra naudojamas simbolinis vykdymas, o ne atsitiktinių skaičių generatorius. / This work consists of three major parts. The first (analytical) part is the review of software quality assurance activities - specifically the testing process. The main focus is on automated test generation. The second (design) part describes the mutation testing based on symbolic execution test generation method. It also specifies the implementation details of the systems under development - described in the static and dynamic perspectives. The third part (research and experimental) is devoted for analysis of developed method. Here wide range of characteristics and metrics are analyzed. Also, some improvements are implemented. This helped to reduce system’s methods cyclomatic complexity and greatly increased speeds at witch tests generation are performed. The method described is characterized by the fact that it can help generate tests that detect mutations in the software code and symbolic execution is used for test generation, rather than a random number generator.

Diplomová práce je zaměřena na stanovení priority testovacích případů s využitím fuzzy logiky. Vhodným přístupem k získání výstupu na základě definovaného vstupu a stanovených pravidel byl zvolen fuzzy model přiřazující prioritu testovacím případům. K dosažení cíle práce byla nejprve stanovena kritéria, parametry a poté určena jejich váha pro jednotlivé testovací případy. Na závěr jsou vyhodnocena vstupní data s využitím řešení v programu MS Excel a MATLAB.

Automated tests have become the most sought after method of testing in today’s market. Automating systems is generally found to be cheaper, faster and more reliable than manually operating them. For this reason many companies decide to make the switch over to this inexpensive yet efficient way of testing, only to realize that their costs have in fact gone up. The reasons for this could be poor planning, automating too much too soon or even automating unnecessary parts of the system. This study takes a closer look at the effects of automating tests used on interlocking systems. A tool has been developed as a part of the automated tests at Bombardier. The main function of the tool is to speed up the testing process by automatically generating test cases rather than having a worker write them down. The aim of this paper is to present this recently developed tool and show how its use significantly reduces testing time, costs and workload compared to the manual tests used today.

The main part in the testing of the software is in the generation of test cases suitable for software system testing. The quality of the test cases plays a major role in reducing the time of software system testing and subsequently reduces the cost. The test cases, in model de- sign stages, are used to detect the faults before implementing it. This early detection offers more flexibility to correct the faults in early stages rather than latter ones. The best of these tests, that covers both static and dynamic software system model specifications, is one of the chal- lenges in the software testing. The static and dynamic specifications could be represented efficiently by Unified Modelling Language (UML) class diagram and sequence diagram. The work in this thesis shows that High Level Petri Nets (HLPN) can represent both of them in one model. Using a proper model in the representation of the software specifications is essential to generate proper test cases. The research presented in this thesis introduces novel and automated test cases generation techniques that can be used within a software sys- tem design testing. Furthermore, this research introduces e cient au- tomated technique to generate a formal software system model (HLPN) from semi-formal models (UML diagrams). The work in this thesis con- sists of four stages: (1) generating test cases from class diagram and Object Constraint Language (OCL) that can be used for testing the software system static specifications (the structure) (2) combining class diagram, sequence diagram and OCL to generate test cases able to cover both static and dynamic specifications (3) generating HLPN automat- ically from single or multi sequence diagrams (4) generating test cases from HLPN. The test cases that are generated in this work covered the structural and behavioural of the software system model. In first two phases of this work, the class diagram and sequence diagram are decomposed to nodes (edges) which are linked by Classes Hierarchy Table (CHu) and Edges Relationships Table (ERT) as well. The linking process based on the classes and edges relationships. The relationships of the software system components have been controlled by consistency checking technique, and the detection of these relationships has been automated. The test cases were generated based on these interrelationships. These test cases have been reduced to a minimum number and the best test case has been selected in every stage. The degree of similarity between test cases is used to ignore the similar test cases in order to avoid the redundancy. The transformation from UML sequence diagram (s) to HLPN facilitates the simpli cation of software system model and introduces formal model rather than semi-formal one. After decomposing the sequence diagram to Combined Fragments, the proposed technique converts each Combined Fragment to the corresponding block in HLPN. These blocks are con- nected together in Combined Fragments Net (CFN) to construct the the HLPN model. The experimentations with the proposed techniques show the effectiveness of these techniques in covering most of the software system specifications.

O teste de software engloba diferentes técnicas, métodos e conceitos capazes de garantir a qualidade dos mais variados tipos de sistemas. Dentre tais técnicas, encontra-se o teste baseado em Máquinas de Estados Finitos (MEFs), que visa a garantir a conformidade entre a implementação e a especificação de um software. Com esse propósito, diversos métodos foram propostos para a geração de seqüências de verificação que garantam cobertura total das possíveis falhas existentes em uma implementação. A maioria dos métodos conhecidos são baseados na utilização de seqüências de distinção. Esse recurso, porem, não existe para toda MEF. Alguns métodos buscam a geração de seqüências de verificação baseados em recursos alternativos as seqüências de distinção, contudo, as seqüências geradas são exponencialmente longas. Este trabalho apresenta um método para geração de seqüências de verificação que visa a reduzir o tamanho das seqüências geradas para o domínio de MEFs que não dispõem de seqüência de distinção. Para isso, o método proposto baseia-se na utilização de conjuntos de distinção. Uma avaliação experimental foi realizada afim de mensurar a redução proporcionada pelo método proposto em relação aos principais métodos existentes na literatura. Com esse intuito, foram geradas MEFs aleatórias sob a perspectiva diferentes fatores. Em relação a variação do número de estados, os resultados indicaram reduções acima de 99; 5% em comparação com os métodos existentes, quando analisadas 75% das MEFs geradas / Software testing involves several techniques, methods, and concepts employed to guarantee a high level of quality in different application domains. Among such techniques, Finite State Machine (FSM) based testing aims to guarantee the conformance between the implementation and the specification of a system under test. In this context, several methods were proposed to generate checking sequences that cover all the possible faults existing in an implementation. Most of these methods are based on a special sequence, named distinguishing sequence, which does not exist for every minimal machine. Some methods were proposed to generate checking sequences based on alternative solutions in order to be applied on FSMs that do not have distinguishing sequences. However, these methods generate checking sequences exponentially long. This work proposes a method to generate checking sequences using identification sets. These sets exist for every minimal FSM and also lead to shorter checking sequences. We conducted an experimental study to compare the proposed method with the main existing methods. In the experiments, we used random FSMs that have different configurations of states, inputs, and outputs. Concerning the variation of number of states, the results show reductions higher than 99:5% in comparison with the existing methods for 75% of the experimented machines

A realização de atividades de teste é indispensável para a garantia da qualidade de um produto e para a identificação de defeitos, diminuindo custos de manutenção e evitando ao máximo o risco do cliente encontrar esses defeitos. Nessa linha, testes baseados em modelos têm se mostrado atrativos, pois o custo de geração de casos de testes e de correção de defeitos tende a ser menor. Devido à sua simplicidade conceitual e expressividade na descrição do comportamento de um sistema, um dos modelos mais usados e pesquisados na área de teste baseado em modelos são as Máquinas de Estados Finitos (MEFs). Por meio de MEFs e com apoio de ferramentas apropriadas, a geração de casos de testes para avaliar os comportamentos esperados de um sistema é automatizada, reduzindo tanto o custo da geração e da manutenção quanto as falhas humanas. Desta forma, a aplicabilidade de métodos de geração de casos de teste baseados em modelos no contexto de sistemas embarcados vem sendo investigada. O objetivo deste trabalho de mestrado consiste em investigar a aplicabilidade dos métodos de geração em cenários de teste reais, com foco em sistemas embarcados, identificando as difi- culdades e limitações do processo, bem como os requisitos essenciais para a adequação dos métodos de geração propostos na literatura e de ferramentas de apoio à atividade de teste. O foco principal do projeto é a implementação de mecanismos que atendam aos requisitos levantados, visando a usabilidade, segurança e portabilidade da ferramenta / Test activities are essential to ensure the quality of products and identify faults to reduce maintenance costs and avoid that the client finds these faults. In this sense, model-based tests have been proved useful, because the cost of generating test cases and fault correction tend to be smaller. Due to its conceptual simplicity and expressiveness in describing the behavior of a system, Finite State Machines (FSM) have been used and researched in the model-based testing area. FSMs, employed with the support of appropriate tools, enable the generation of test cases in an automated way to assess the expected behavior of a system, reducing both the generation and maintenance costs and human failures. Thus, the applicability of test cases generation methods based on models in the context of embedded systems should be investigated. Test cases generation methods based on FSM are designed to derive test cases from the model. In this context, this work aims to investigate the applicability of generation methods in real-world scenarios, focusing embedded systems. It should identify the difficulties and limitations of the process, as well as the essential requirements for the adequacy of generation methods proposed in the literature and tools to support the test activity. The main focus of the project is the implementation of mechanisms that meet the elicited requirements in order to provide usability, security and tool portability