Made available in DSpace on 2014-12-17T15:48:00Z (GMT). No. of bitstreams: 1 ErnestoCBM_DISSERT.pdf: 1152535 bytes, checksum: a61c509f155d27fa9ab04bc69c4607e8 (MD5) Previous issue date: 2012-02-10 / Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior / Formal methods and software testing are tools to obtain and control software quality. When used together, they provide mechanisms for software specification, verification and error detection. Even though formal methods allow software to be mathematically verified, they are not enough to assure that a system is free of faults, thus, software testing techniques are necessary to complement the process of verification and validation of a system. Model Based Testing techniques allow tests to be generated from other software artifacts such as specifications and abstract models. Using formal specifications as basis for test creation, we can generate better quality tests, because these specifications are usually precise and free of ambiguity. Fernanda Souza (2009) proposed a method to define test cases from B Method specifications. This method used information from the machine s invariant and the operation s precondition to define positive and negative test cases for an operation, using equivalent class partitioning and boundary value analysis based techniques. However, the method proposed in 2009 was not automated and had conceptual deficiencies like, for instance, it did not fit in a well defined coverage criteria classification. We started our work with a case study that applied the method in an example of B specification from the industry. Based in this case study we ve obtained subsidies to improve it. In our work we evolved the proposed method, rewriting it and adding characteristics to make it compatible with a test classification used by the community. We also improved the method to support specifications structured in different components, to use information from the operation s behavior on the test case generation process and to use new coverage criterias. Besides, we have implemented a tool to automate the method and we have submitted it to more complex case studies / M?todos formais e testes s?o ferramentas para obten??o e controle de qualidade de software. Quando utilizadas em conjunto, elas prov?em mecanismos para especifica??o, verifica??o e detec??o de falhas de um software. Apesar de permitir que sistemas sejam matematicamente verificados, m?todos formais n?o s?o suficientes pra garantir que um sistema esteja livre de defeitos, logo, t?cnicas de teste de software s?o necess?rias para completar o processo de verifica??o e valida??o de um sistema. T?cnicas de Testes Baseados em Modelos permitem que testes sejam gerados a partir de outros artefatos de software como especifica??es e modelos abstratos. Ao utilizarmos especifica??es formais como base para a cria??o de testes, podemos gerar testes de melhor qualidade pois estas especifica??es costumam ser precisas e livres de ambiguidade. Fernanda Souza (2009) prop?s um m?todo para definir casos de teste a partir de especifica??es do M?todo B. Este m?todo utilizava informa??es do invariante de uma m?quina e das pr?-condi??es de uma opera??o para definir casos de teste positivos e negativos para tal opera??o, atrav?s de t?cnicas baseadas em particionamento em classes de equival?ncia e an?lise de valor limite. No entanto, a proposta de 2009 n?o inclu?a automa??o e possu?a algumas defici?ncias conceituais como, por exemplo, n?o se encaixar exatamente em uma classifica??o de crit?rios de cobertura bem definida. Iniciamos nosso trabalho com um estudo de caso que aplicou o m?todo a um exemplo de especifica??o B proveniente da ind?stria. A partir deste estudo obtivemos subs?dios para o aperfei?o?-lo. Em nosso trabalho aperfei?oamos o m?todo proposto, reescrevendo e adicionando caracter?sticas para torn?-lo compat?vel com uma classifica??o de testes utilizada pela comunidade. O m?todo tamb?m foi melhorado para suportar especifica??es estruturadas em v?rios componentes, utilizar informa??es sobre o comportamento da opera??o durante a cria??o de casos de teste e utilizar novos crit?rios de cobertura. Al?m disso, implementamos uma ferramenta para automatiz?-lo e o submetemos a estudos de caso mais complexos

Submitted by Johnny Rodrigues (johnnyrodrigues@ufcg.edu.br) on 2018-08-16T14:52:23Z No. of bitstreams: 1 CAMILA DE LUNA MACIEL - DISSERTAÇÃO PPGCC 2010..pdf: 7464974 bytes, checksum: 6d1dbd48857a3fa1c75047b6ab0a2016 (MD5) / Made available in DSpace on 2018-08-16T14:52:23Z (GMT). No. of bitstreams: 1 CAMILA DE LUNA MACIEL - DISSERTAÇÃO PPGCC 2010..pdf: 7464974 bytes, checksum: 6d1dbd48857a3fa1c75047b6ab0a2016 (MD5) Previous issue date: 2010-08-06 / CNPq / Dentro da Engenharia de Software, novos paradigmas de desenvolvimento vêm surgindo no intuito de oferecer uma maior produtividade sem perda de qualidade aos softwares desenvolvidos. Um desses paradigmas é o MDD (Model-Driven Development), cuja principal finalidade é a introdução de modelos rigorosos durante todo o processo de desenvolvimento de software oferecendo, dentre outras vantagens, a geração automática de código a partir dos modelos. Contudo, mesmo em processos de desenvolvimento que seguem este paradigma, a atividade de teste de software ainda é fundamental, principalmente teste de integração, cujo objetivo é verificar que os componentes do software, implementados e testados individualmente, provêem a funcionalidade pretendida quando colocados para interagir uns com os outros. Embora classes individuais possam funcionar corretamente, várias novas faltas podem surgir quando os componentes são integrados. No entanto, em teste de integração, dependendo da complexidade do sistema, o número de casos de teste pode ser muito grande. Nesse contexto, o uso de padrões de teste, ou seja, estratégias que já foram utilizadas e se mostraram efetivas em teste de software, pode guiar a escolha de casos de teste mais efetivos e adequados dentre um número muito grande de possíveis casos de teste. Este trabalho tem como objetivo principal fornecer uma nova abordagem de teste de integração, definida dentro de um processo integrado de desenvolvimento e teste dirigidos por modelos (MDD/MDT - Model-Driven Testing), para a geração automática de casos de teste a partir de modelos, utilizando padrões de teste como base para o processo de geração. Para automatizar este processo, foi desenvolvida uma ferramenta baseada em transformações entre modelos segundo práticas da MDA (Model-DrivenArchitecture). Além disso, a abordagem proposta utiliza o perfil de teste da UML para a documentação de todos os artefatos de teste gerados. Adicionalmente, estudos experimentais preliminares foram realizados no intuito de avaliar a abordagem e, consequentemente, a ferramenta de suporte desenvolvida. / Within the Software Engineering, new development paradigms are emerging in order to offer greater productivity without sacrificing quality to the developed software. MDD (Model-Driven Development) is one of these paradigms, whose main purpose is to introduce rigorous models along all the software development process offering, among other advantages, automatic code generation from models. However, even in development processes that follow this paradigm, the software testing activity is still essential, especially integration testing, whose purpose is to verify that the software components, implemented and tested separately, provide the desired functionality when placed to interact with each other. While individual components may function correctly, several new faults can arise when the components are integrated. However, in integration testing, depending on the system complexity, the number of test cases can be very large. In this context, the use of test patterns, i. e., strategies that have been used and proved effective in software testing, can guide the user at choosing test cases more effective and appropriate among a very large number of possible cases test. The main objective of this work is to propose a new approach to integration testing, defined with in an integrated model driven development and test process (MDD/MDT - Model-Driven Testing) for automatically generating test case from models adopting test patterns as basis for the generation process. To automate this process, we have developed a tool based on model transformations according to MDA (Model-Driven Architecture) practices. Furthermore, the proposal approach uses the UML testing profile to document all generated test artifacts. Additionally, preliminary experimental case studies were performed in order to evaluate the proposed approach and hence the developed tool support.

Submitted by Lucienne Costa (lucienneferreira@ufcg.edu.br) on 2018-05-02T20:48:28Z No. of bitstreams: 1 ALAN KELON OLIVEIRA DE MORAES – TESE (PPGCC) 2017.pdf: 2206784 bytes, checksum: 9e05994ae273d28b9871d8d16769dac8 (MD5) / Made available in DSpace on 2018-05-02T20:48:28Z (GMT). No. of bitstreams: 1 ALAN KELON OLIVEIRA DE MORAES – TESE (PPGCC) 2017.pdf: 2206784 bytes, checksum: 9e05994ae273d28b9871d8d16769dac8 (MD5) Previous issue date: 2017-08-31 / O teste baseado em modelos é uma abordagem de teste de software que usa modelos abstratos de uma aplicação para gerar, executar e avaliar os testes. A geração de casos de testes exerce um papel importante no teste baseado em modelos. Como essa geração consiste na busca sistemática por casos de testes que possam ser extraídos dos modelos, o teste baseado em modelos geralmente produz suítes de testes que são caras demais para serem executadas completamente. Técnicas de redução de suítes de testes têm sido propostas para abordar este problema. O objetivo dessas técnicas é obter suítes de testes reduzidas que são mais baratas de serem executadas e tão efetivas na detecção de faltas quanto as suítes completas, dado que as suítes reduzidas mantém o mesmo nível de cobertura, definido por um critério de adequação de testes, da suíte completa. Esses critérios definem que partes do sistema serão testados, com que frequência e sob quais circunstâncias. Entretanto, pouca atenção tem sido dada ao impacto que a escolha do critério tem na redução de suítes de testes. Por outro lado, sistemas de tempo-real são sistemas reativos cujos comportamentos são restringidos pelo tempo. Consequentemente, faltas relacionadas ao tempo são específicas desses sistemas. Para lidar com isso, modelos para sistemas de tempo real devem trabalhar com tempo e, consequentemente, há critérios de adequação de testes específicos para eles. Contudo, a pesquisa sobre redução de suítes de testes não tem focado em sistemas de tempo-real, portanto o impacto de critérios de adequação de testes na redução de suítes é desconhecido. Nesta pesquisa de doutorado objetivamos investigar a influência de critérios de adequação de testes nos resultados da redução de suítes de testes no contexto de teste baseado em modelos de sistemas de tempo-real. Em particular, nós estamos interessados no modelo Timed Input-Output Symbolic Transition Systems (TIOSTS), porque ele é um modelo de sistema de transições no qual dados e tempo são definidos simbolicamente, já que sistemas de transição são a base para o teste de conformidade de sistemas de tempo real. Para alcançar o objetivo da pesquisa, primeiramente, nós definimos 19 critérios de adequação de testes para o modelo TIOSTS. Os critérios definidos incluem critérios baseados em transições, fluxo de dados e tempo. Depois nós formalizamos uma hierarquia com esses critérios, onde eles estão parcialmente ordenados pela relação de inclusão estrita. Segundamente, nós avaliamos empiricamente o custo-benefício de doze dos critérios definidos e cinco técnicas de redução de suítes de testes. Nós avaliamos o tamanho, o tempo de execução e a detecção de faltas das suítes de testes reduzidas de cada uma das 60 combinações de critério e técnica. No experimento, nós usamos modelos de especificação, em TIOSTS, de uma máquina de recarga de cartão do metrô, de um sistema de alarme anti-roubo e de um limitador automático de velocidade de carros. Além disso, usamos simulações das implementações, que geram rastros corretos para os modelos. Por fim, o teste de mutação foi usado para gerar mutantes dos modelos de especificação, que, por sua vez, foram traduzidos para simulações com a finalidade de simular modelos de implementações defeituosas. As evidências empíricas sugerem que os critérios de adequação de testes mais próximos do topo da hierarquia produziram suítes reduzidas com melhor custo-benefício com relação à detecção de faltas e tempo de execução. Com relação às técnicas de redução, a técnica aleatória obteve melhor custo-benefício dentre as técnicas avaliadas. Os resultados apontam que os critérios explicam mais a variação nos resultados do que as técnicas. / Model-based testing is a testing approach that relies on the existence of abstract models of an application to generate, execute and evaluate tests. Test case generation plays an important role in model-based testing. Since it consists of a systematic search for test cases that can be extracted from models, model-based testing usually generates large test suites which are too expensive to execute in full. Test suite reduction techniques have been proposed to address this problem. The goal of the techniques is to obtain reduced test suites that are both cheaper to execute and as effective at detecting faults as the original suite, given that the reduced test suites maintain the same coverage level of the complete test suite required by a test adequacy criterion. These criteria define which parts of the system are going to be tested, how often and under what circumstances. Nevertheless, little attention has been paid to the impact of the criterion choice in test suite reduction research. On the other hand, real-time systems are reactive systems whose behavior is constrained by time. Consequently, time-related faults are specific to these systems. In order to cope with this issue, models for real-time systems must deal with time and, consequently, there are specific test adequacy criteria for them. However, test suite reduction research has not focused on real-time systems, therefore the impact of test adequacy criteria for models of real-time systems on test suite reduction is unknown. In this doctoral research, we aim at investigating the influence of test adequacy criteria on the outcomes of test suite reduction techniques in the context of model-based testing of real-time systems. In particular, we are interested in the Timed Input-Output Symbolic Transition Systems (TIOSTS) model because it is an expressive transition system in which data and time are symbolically defined, and transition systems are the basis for conformance testing of real-time systems. In order to achieve the research objective, first, we defined 19 test adequacy criteria for TIOSTS models. The defined criteria include transition-based criteria, data-flow-oriented criteria and real-time systems criteria. Next, we formalized a hierarchy with these criteria which is partially ordered by the strict inclusion relation. Second, we evaluated the cost-effectiveness of twelve criteria and five test suite reduction techniques in empirical studies of test suite reduction. We evaluated the size, execution time and fault detection of reduced test suites obtained from each combination of criterion and technique. In the experiment, we used TIOSTS specification models of a refilling machine for charging the subway card, a burglar alarm system, and an automated car speed limiter; simulations of the implementations, which generate correct traces for the models; and mutation testing to generate mutants of the specification models, which were also translated to simulations in order to simulate faulty model implementations. Empirical evidence suggests that test adequacy criteria closer to the top of the family obtained reduced test suites with better costeffectiveness regarding fault detection and execution time. With respect to the test suite reduction techniques, the Random technique obtained better cost-effectiveness among the evaluated criteria. Results also suggests that the criteria explain more the variation in fault detection and execution time of reduced test suites than the techniques.

A Arquitetura Orientada a Serviços (SOA) é um estilo arquitetural formado por um conjunto de restrições que visa promover a escalabilidade e a flexibilidade de um sistema, provendo suas funcionalidades como serviços. Nos últimos anos, um estilo alternativo foi proposto e amplamente adotado, que projeta as funcionalidades de um sistema como recursos. Este estilo arquitetural orientado a recursos é chamado de REST. O teste de serviços web em geral apresenta vários desafios devido a sua natureza distribuída, canal de comunicação pouco confiável, baixo acoplamento e a falta de uma interface de usuário. O teste de serviços RESTful (serviços que utilizam o REST) compartilham estes mesmos desafios e ainda necessitam que suas restrições sejam obedecidas. Estes desafios demandam testes mais sistemáticos e formais. Neste contexto, o teste baseado em modelos (TBM) se apresenta como um processo viável para abordar essas necessidades. O modelo que representa o sistema deve ser simples e ao mesmo tempo preciso para que sejam gerados casos de teste com qualidade. Com base nesse contexto, este projeto de mestrado propõe uma abordagem baseada em modelos para testar serviços RESTful. O modelo comportamental adotado foi a máquina de estados de protocolos UML, capaz de formalizar a interface do serviço enquanto esconde o seu funcionamento interno. Uma ferramenta foi desenvolvida para gerar automaticamente os casos de teste usando critérios de cobertura de estados e transições para percorrer o modelo / Service Oriented Architecture (SOA) is an architectural style consisting of a set of restrictions aimed at promoting the scalability and flexibility of a system, providing its functionalities as services. In recent years, an alternative style was proposed and widely adopted, which designs the system\'s functionalities as resources. This resource oriented architectural style is called REST. In general, the test of web services has several challenges due to its distributed nature, unreliable communication channel, low coupling and the lack of a user interface. Testing RESTful web services (services that use REST) share these same challenges and also need to obey the REST constraints. These challenges require a more systematic and formal testing approach. In this context, model based testing presents itself as a viable process for addressing those needs. The model that represents the system should be simple and precise enough to generate quality test cases. Based on this context, this work proposes a model based approach to test RESTful web services. The behavioral model used was the UML protocol state machine, which is capable to provide a formalization of the service interface, while hiding its internal behaviour. A tool was developed to automatically generate test cases using the state and transition coverage criteria to traverse the model

SERVICE oriented architecture (SOA) is an architectural style to structure software systems, fostering loose coupling and dynamic integration among the applications. The use of SOA and Web services to develop complex and large business processes demands more formal and systematic testing. In addition, characteristics of this type of software limit the straightforward application of traditional testing techniques. Model-based testing (MBT) is a promising approach to deal with these problems. This dissertation investigates how two modeling techniques, namely Finite State Machine (FSM) and Event Sequence Graph (ESG), can be used to support MBT of service-oriented applications. Both techniques model different aspects and can be applied in a complementary way. Initially, we define an MBT process for service-oriented applications that employs FSMs. Based on previous experience, we propose a model-based approach to test composite services using ESGs. This approach is holistic, once test suites are generated to cover both desired situations (positive testing) and unexpected behaviors (negative testing). Three experimental studies evaluate the proposed approach: (i) a case study, (ii) a cost analysis, and (iii) a study in industry. Testing tools are also presented to support its practical use / A Arquitetura orientada a serviço (SOA) é um estilo arquitetural para estruturar sistemas de software de modo que exista um baixo grau de acoplamento entre as aplicações e essas possam ser facilmente integradas de forma dinâmica. A incorporação de SOA e serviços Web em sistemas que modelam processos de negócios grandes e complexos contribui para a necessidade de testes mais formais e sistemáticos. Além disso, características próprias dessa nova classe de software fazem com que técnicas de teste tradicionais não possam ser diretamente aplicadas. O teste baseado em modelo (TBM) apresenta-se como uma abordagem promissora que busca a resolução desses problemas. Esta tese investiga como duas técnicas de modelagem, Máquina de Estados Finitos (MEF) e Grafo de Sequência de Eventos (GSE), podem ser utilizadas para apoiar o TBM de aplicações orientadas a serviço. Essas técnicas modelam diferentes aspectos e podem ser aplicadas de forma complementar. Inicialmente, é definido um processo de TBM para aplicações orientadas a serviço que emprega MEFs. Com base na experiência adquirida, é proposta uma abordagem baseada em modelo para o teste de serviços compostos usando GSEs. Essa abordagem é holística uma vez que conjuntos de teste são gerados para cobrir tanto situações desejadas (teste positivo) quanto comportamentos inesperados (teste negativo). Três estudos experimentais avaliam a abordagem proposta: (i) um estudo de caso, (ii) uma análise de custo e (ii) um estudo na indústria. Ferramentas de teste também são apresentadas para apoiar o uso prático da abordagem proposta

For many companies today, it is highly valuable to collect and analyse data in order to support decision making and functions of various sorts. However, this kind of data-driven approach is seldomly applied to software testing and there is often a lack of verification that the testing performed is relevant to how the system under test is used. Therefore, the aim of this thesis is to investigate the possibility of introducing a data-driven approach to test automation by extracting user behaviour data and curating it to form input for testing. A prestudy was initially conducted in order to collect and assess different data sources for augmenting the testing. After suitable data sources were identified, the required data, including data about user activity in the system, was extracted. This data was then processed and three prototypes where built on top of this data. The first prototype augments the model-based testing by automatically creating models of the most common user behaviour by utilising data mining algorithms. The second prototype tests the most frequent occurring client actions. The last prototype visualises which features of the system are not covered by automated regression testing. The data extracted and analysed in this thesis facilitates the understanding of the behaviour of the users in the system under test. The three prototypes implemented with this data as their foundation can be used to assist other testing methods by visualising test coverage and executing regression tests.

La vérification de la résistance aux attaques des implémentations embarquées des vérifieurs de code intermédiaire Java Card est une tâche complexe. Les méthodes actuelles n'étant pas suffisamment efficaces, seule la génération de tests manuelle est possible. Pour automatiser ce processus, nous proposons une méthode appelée VTG (Vulnerability Test Generation, génération de tests de vulnérabilité). En se basant sur une représentation formelle des comportements fonctionnels du système sous test, un ensemble de tests d'intrusions est généré. Cette méthode s'inspire des techniques de mutation et de test à base de modèle. Dans un premier temps, le modèle est muté selon des règles que nous avons définies afin de représenter les potentielles attaques. Les tests sont ensuite extraits à partir des modèles mutants. Deux modèles Event-B ont été proposés. Le premier représente les contraintes structurelles des fichiers d'application Java Card. Le VTG permet en quelques secondes de générer des centaines de tests abstraits. Le second modèle est composé de 66 événements permettant de représenter 61 instructions Java Card. La mutation est effectuée en quelques secondes. L'extraction des tests permet de générer 223 tests en 45 min. Chaque test permet de vérifier une précondition ou une combinaison de préconditions d'une instruction. Cette méthode nous a permis de tester différents mécanismes d'implémentations de vérifieur de code intermédiaire Java Card. Bien que développée pour notre cas d'étude, la méthode proposée est générique et a été appliquée à d'autres cas d'études. / Verification of the resistance of attacks against embedded implementations of the Java Card bytecode verifiers is a complex task. Current methods are not sufficient, only the generation of manual testing is possible. To automate this process, we propose a method called VTG (Vulnerability Test Generation). Based on a formal representation of the functional behavior of the system under test, a set of intrusion test is generated. This method is based on techniques of mutation and model-based testing. Initially, the model is transferred according to rules that we have defined to represent potential attacks. The tests are then extracted from the mutant models. Two Event-B models have been proposed. The first represents the structural constraints of the Java Card application files. The VTG allows in seconds to generate hundreds of abstract tests. The second model is composed of 66 events to represent 61 Java Card instructions. The mutation is effected in a few seconds. Extraction tests to generate 223 test 45 min. Each test checks a precondition or a combination of preconditions of a statement. This method allowed us to test different implementations of mechanisms through Java Card bytecode verifier. Although developed for our case study, the proposed method is generic and has been applied to other case studies.

Les travaux présentés dans cette thèse proposent une méthode de génération automatique de tests à partir de modèles.Cette méthode emploie deux langages de modélisations UML4MBT et OCL4MBT qui ont été spécifiquement dérivées d’ UML et OCL pour la génération de tests. Ainsi les comportements, la structure et l’état initial du système sont décrits au travers des diagrammes de classes, d’objets et d’états-transitions.Pour générer des tests, l’évolution du modèle est représente sous la forme d’un système de transitions. Ainsi la construction de tests est équivalente à la découverte de séquences de transitions qui relient l’´état initial du système à des états validant les cibles de test.Ces séquences sont obtenues par la résolution de scénarios d’animations par des prouveurs SMT et solveurs CSP. Pour créer ces scénarios, des méta-modèles UML4MBT et CSP4MBT regroupant formules logiques et notions liées aux tests ont été établies pour chacun des outils.Afin d’optimiser les temps de générations, des stratégies ont été développé pour sélectionner et hiérarchiser les scénarios à résoudre. Ces stratégies s’appuient sur la parallélisation, les propriétés des solveurs et des prouveurs et les caractéristiques de nos encodages pour optimiser les performances. 5 stratégies emploient uniquement un prouveur et 2 stratégies reposent sur une collaboration du prouveur avec un solveur.Finalement l’intérêt de cette nouvelle méthode à été validée sur des cas d’études grâce à l’implémentation réalisée. / This thesis describes an automatic test generation process from models.This process uses two modelling languages, UML4MBT and OCL4MBT, created specificallyfor tests generation. Theses languages are derived from UML and OCL. Therefore the behaviours,the structure and the initial state of the system are described by the class diagram, the objectdiagram and the state-chart.To generate tests, the evolution of the model is encoded with a transition system. Consequently,to construct a test is to find transition sequences that rely the initial state of the system to thestates described by the test targets.The sequence are obtained by the resolution of animation scenarios. This resolution is executedby SMT provers and CSP solvers. To create the scenario, two dedicated meta-models, UML4MBTand CSP4MBT have been established. Theses meta-models associate first order logic formulas withthe test notions.7 strategies have been developed to improve the tests generation time. A strategy is responsiblefor the selection and the prioritization of the scenarios. A strategy is built upon the properties ofthe solvers and provers and the specification of our encoding process. Moreover the process canalso be paralleled to get better performance. 5 strategies employ only a prover and 2 make theprover collaborate with a solver.Finally the interest of this process has been evaluated through a list of benchmark on variouscases studies.

O Teste Baseado em Modelos (TBM) emergiu como uma estratégia promissora para minimizar problemas relacionados à falta de tempo e recursos em teste de software e visa verificar se a implementação sob teste está em conformidade com sua especificação. Casos de teste são gerados automaticamente a partir de modelos comportamentais produzidos durante o ciclo de desenvolvimento de software. Entre as técnicas de modelagem existentes, Sistemas de Transição com Entrada/Saída (do inglês, Input/Output Transition Systems - IOTSs), são modelos amplamente utilizados no TBM por serem mais expressivos do que Máquinas de Estado Finito (MEFs). Apesar dos métodos existentes para geração de testes a partir de IOTSs, o problema da seleção de casos de testes é um tópico difícil e importante. Os métodos existentes para IOTS são não-determinísticos, ao contrário da teoria existente para MEFs, que fornece garantia de cobertura completa com base em um modelo de defeitos. Esta tese investiga a aplicação de modelos de defeitos em métodos determinísticos de geração de testes a partir de IOTSs. Foi proposto um método para geração de conjuntos de teste com base no método W para MEFs. O método gera conjuntos de teste de forma determinística além de satisfazer condições de suficiência de cobertura da especificação e de todos os defeitos do domínio de defeitos definido. Estudos empíricos avaliaram a aplicabilidade e eficácia do método proposto: resultados experimentais para analisar o custo de geração de conjuntos de teste utilizando IOTSs gerados aleatoriamente e um estudo de caso com especificações da indústria mostram a efetividade dos conjuntos gerados em relação ao método tradicional de Tretmans. / Model-Based Testing (MBT) has emerged as a promising strategy for the minimization of problems related to time and resource limitations in software testing and aims at checking whether the implementation under test is in compliance with its specification. Test cases are automatically generated from behavioral models produced during the software development life cycle. Among the existing modeling techniques, Input/Output Transition Systems (IOTSs) have been widely used in MBT because they are more expressive than Finite State Machines (FSMs). Despite the existence of test generation methods for IOTSs, the problem of selection of test cases is an important and difficult topic. The current methods for IOTSs are non-deterministic, in contrast to the existing theory for FSMs that provides complete fault coverage guarantee based on a fault model. This manuscript addresses the application of fault models to deterministic test generation methods from IOTSs. A method for the test suite generation based on W method for FSMs is proposed for IOTSs. It generates test suites in a deterministic way and also satisfies sufficient conditions of specification coverage and all faults in a given fault domain. Empirical studies evaluated its applicability and effectiveness. Experimental results for the analyses of the cost of test suite generation by random IOTSs and a case study with specifications from the industry show the effectiveness of the test suites generated in relation to the traditional method of Tretmans.

Autonomy is the most demanded yet hard-to-achieve feature of recent and future software systems. Self-driving cars, mail-delivering drones, automated guided vehicles in production sites, and housekeeping robots need to decide autonomously during most of their operation time. As soon as human intervention becomes necessary, the cost of ownership increases, and this must be avoided. Although the algorithms controlling autonomous systems become more and more intelligent, their hardest opponent is their inflexibility. The more environmental situations such a system is confronted with, the more complexity the control of the autonomous system will have to master. To cope with this challenge, engineers have approached a system design, which adopts feedback loops from nature. The resulting architectural principle, which they call self-adaptive systems, follows the idea of iteratively gathering sensor data, analyzing it, planning new adaptations of the system, and finally executing the plan. Often, adaptation means to alter the system setup, re-wire components, or even exchange control algorithms to keep meeting goals and requirements in the newly appeared situation. Although self-adaptivity helps engineers to organize the vast amount of information in a self-deciding system, it remains hard to deal with the variety of contexts, which involve both environmental influences and knowledge about the system\'s internals. This challenge not only holds for the construction phase but also for verification and validation, including software test. To assure sufficient quality of a system, it must be tested under an enormous and, thus, unmanageable, number of different contextual situations and manual test-cases. This thesis proposes a novel set of methods and model types, which help test engineers to specify precisely what they expect from a self-adaptive system under test. The formal nature of the introduced artifacts allows for automatically generating test-suites or running simulations in the loop so that a qualitative verdict on the system\'s correctness can be gained. Additional to these conceptional contributions, the thesis describes a model-based adaptivity test environment, which test engineers can use for testing actual self-adaptive systems. The implementation includes comprehensive tooling for creating the introduced types of models, generating test-cases, simulating them in the loop, automating tests, and reporting. Composing all enabling components for these tasks constitutes a reference architecture of integrated test environments for self-adaptive systems. We demonstrate the completeness and accuracy of the technical approach together with the underlying concepts by evaluating them in an experimental case study where an autonomous robot interacts with human co-workers. In summary, this thesis proposes concepts for automatically and, thus, efficiently testing self-adaptive systems. The quality, which is fostered by this novel approach, is resilience: the ability of a system to maintain its promises while facing changing environments.:1 Introduction 1 1.1 Problem Description 1 1.2 Overview of Adopted Methods 3 1.3 Hypothesis and Main Contributions 4 1.4 Organization of This Thesis 5 I Foundations 7 2 Background 9 2.1 Self-adaptive Software and Autonomic Computing 9 2.1.1 Common Principles and Components of SAS 10 2.1.2 Concrete Implementations and Applications of SAS 12 2.2 Model-based Testing 13 2.2.1 Testing for Dependability 14 2.2.2 The Basics of Testing 15 2.2.3 Automated Test Design 18 2.3 Dynamic Variability Management 22 2.3.1 Software Product Lines 23 2.3.2 Dynamic Software Product Lines 25 3 Related Work: Existing Research on Testing Self-Adaptive Systems 29 3.1 Testing Context-Aware Applications 30 3.2 The SimSOTA Project 31 3.3 Dynamic Variability in Complex Adaptive Systems (DiVA) 33 3.4 Other Early-Stage Research 34 3.5 Taxonomy of Requirements of Model-based SAS Testing 36 II Methods 39 4 Model-driven SAS Testing 41 4.1 Problem/Solution Fit 41 4.2 Example: Surveillance Drone 43 4.3 Concepts and Models for Testing Self-Adaptive Systems 44 4.3.1 Test Case Generation vs. Simulation in the Loop 44 4.3.2 Incremental Modeling Process 45 4.3.3 Basic Representation Format: Petri Nets 46 4.3.4 Context Variation 50 4.3.5 Modeling Adaptive Behavior 53 4.3.6 Dynamic Context Change 57 4.3.7 Interfacing Context from Behavioral Representation 62 4.3.8 Adaptation Mode Variation 64 4.3.9 Context-Dependent Recon guration 67 4.4 Adequacy Criteria for SAS Test Models 71 4.5 Discussion on the Viability of the Employed Models 71 4.6 Comparison to Related Work 73 4.7 Summary and Discussion 74 5 Model-based Adaptivity Test Environment 75 5.1 Technological Foundation 76 5.2 MATE Base Components 77 5.3 Metamodel Implementation 78 5.3.1 Feature-based Variability Model 79 5.3.2 Abstract and Concrete Syntax for Textual Notations 80 5.3.3 Adaptive Petri Nets 86 5.3.4 Stimulus and Recon guration Automata 87 5.3.5 Test Suite and Report Model 87 5.4 Test Generation Framework 87 5.5 Test Automation Framework 91 5.6 MATE Tooling and the SAS Test Process 93 5.6.1 Test Modeling 94 5.6.2 Test Case Generation 95 5.6.3 Test Case Execution and Test Reporting 96 5.6.4 Interactive Simulation Frontend 96 5.7 Summary and Discussion 97 III Evaluation 99 6 Experimental Study: Self-Adaptive Co-Working Robots 101 6.1 Robot Teaching and Co-Working with WEIR 103 6.1.1 WEIR Hardware Components 104 6.1.2 WEIR Software Infrastructure 105 6.1.3 KUKA LBR iiwa as WEIR Manipulator 106 6.1.4 Self-Adaptation Capabilities of WEIR 107 6.2 Cinderella as Testable Co-Working Application 109 6.2.1 Cinderella Setup and Basic Functionality 109 6.2.2 Co-Working with Cinderella 110 6.3 Testing Cinderella with MATE 112 6.3.1 Automating Test Execution 112 6.3.2 Modeling Cinderella in MATE 113 6.3.3 Testing Cinderella in the Loop 121 6.4 Evaluation Verdict and Summary 123 7 Summary and Discussion 125 7.1 Summary of Contributions 126 7.2 Open Research Questions 127 Bibliography 129 Appendices 137 Appendix Cinderella De nitions 139 1 Cinderella Adaptation Bounds 139 2 Cinderella Self-adaptive Workflow 140