Die Ansprüche von Kunden an neue (Software-)Produkte wachsen stetig. Produkte sollen genau auf die einzelnen Kundenwünsche zugeschnitten sein, sodass der Kunde genau die Funktionalität erhält und bezahlt die er benötigt. Hersteller reagieren auf diese gestiegenen Ansprüche mit immer mehr Varianten in denen sie ihre Produkte ihren Kunden anbieten. Die Variantenvielfalt hat in solchem Maß zugenommen, dass selbst in Massen gefertigte Produkte heute als Unikate produziert werden können. Neue Methoden wie Produktlinienentwicklung unterstützen die Entwicklung solcher variantenreicher Systeme. Während der Aufwand für die Entwicklung neuer Varianten nun sinkt, profitiert die Qualitätssicherung nicht vom Effizienzgewinn der Entwicklung. Im Gegenteil: Insbesondere beim Test wird zunächst jede Variante wie ein einzelnes Produkt behandelt. Bei variantenreichen Systemen ist dies aufwandsbedingt jedoch nicht mehr möglich. Die in dieser Arbeit vorgestellten Testentwurfsmethoden berücksichtigen die Variantenvielfalt in besonderem Maße. Bisher wurden, nach einer Stichprobenauswahl zur Reduktion des Testaufwands, die Testfälle auf Basis der konkreten Produkte entworfen. Statt nun auf Basis konkreter Produkte werden in dieser Arbeit zwei Ansätze vorgestellt, die die Phase des Testentwurfs auf die Produktlinienebene heben. Die bei Anwendung dieser Methoden entstehenden Testfälle enthalten, je nach Inhalt, Freiheitsgrade bzgl. ihrer Anforderungen an eine Variante, sodass ein Testfall auf ein oder mehrere Varianten angewendet wird. Ausgehend von solchen Testfällen werden in dieser Arbeit neue Kriterien zur Stichprobenauswahl entwickelt. Mit diesen Kriterien kann der Umfang der Stichprobe, aber auch Eigenschaften der zu testenden Varianten bzgl. eines gegebenes Testziel optimiert werden. So ist es möglich, z.B. sehr wenige oder sehr unterschiedliche Varianten zum Test auszuwählen. Insgesamt werden in dieser Arbeit fünf Kriterien definiert und auf ihr Fehleraufdeckungspotenzial untersucht. Zu diesem Zweck werden neue Bewertungskriterien zur Fehleraufdeckungswahrscheinlichkeit von Produktlinientests etabliert. Somit ist erstmalig eine quantitative sowie qualitative Bewertung von Produktlinientests möglich. Die Ergebnisse der vorgestellten Methoden und Auswahlkriterien werden sowohl untereinander evaluiert, als auch konventionellen Testmethoden für Produktliniensysteme gegenübergestellt. An vier Beispielen unterschiedlicher Gro{\"ss}e werden die in dieser Arbeit vorgestellten Methoden evaluiert. / Consumer expectations of (software-)products are growing continuously. They demand products that fit their exact needs, so they pay only for necessary functionalities. Producers react to those demands by offering more variants of a product. Product customization has reached a level where classically mass produced goods, like cars, can be configured to unique items. New paradigms facilitate the engineering of such variant-rich systems and reduce costs for development and production. While development and production became more efficient, quality assurance suffers from treating each variant as a distinct product. In particular, test effort is affected, since each variant must be tested sufficiently prior to production. For variant-rich systems this testing approach is not feasible anymore. The methods for test design presented in this thesis overcome this issue by integrating variability into the test design process. The resulting test cases include requirements for variants, which must be fulfilled to execute the test successfully. Hence multiple variants may fulfill these requirements, each test case may be applicable to more than only one variant. Having test cases with requirements enables sampling subsets of variants for the purpose of testing. Under the assumption that each test case must be executed once, variants can be sampled to meet predefined test goals, like testing a minimal or diverse subset of variants. In this thesis, five goals are defined and evaluated by assessing the tests for their fault detection potential. For this purpose, new criteria for assessing the fault detection capability of product line tests are established. These criteria enable quantitative as well as qualitative assessment of such test cases for the first time. The results of the presented methods are compared with each other and furthermore with state of the art methods for product line testing. This comparison is carried out on four examples of different sizes, from small to industry-grade.

Access Control (AC) is a major pillar in software security. In short, AC ensures that only intended users can access resources and only the required access to accomplish some task will be given. In this context, Role Based Access Control (RBAC) has been established as one of the most important paradigms of access control. In an organization, users receive responsibilities and privileges through roles and, in AC systems implementing RBAC, permissions are granted through roles assigned to users. Despite the apparent simplicity, mistakes can occur during the development of RBAC systems and lead to faults or either security breaches. Therefore, a careful verification and validation process becomes necessary. Access control testing aims at showing divergences between the actual and the intended behavior of access control mechanisms. Model Based Testing (MBT) is a variant of testing that relies on explicit models, such as Finite State Machines (FSM), for automatizing test generation. MBT has been successfully used for testing functional requirements; however, there is still lacking investigations on testing non-functional requirements, such as access control, specially in test criteria. In this Master Dissertation, two aspects of MBT of RBAC were investigated: FSM-based testing methods on RBAC; and Test prioritization in the domain of RBAC. At first, one recent (SPY) and two traditional (W and HSI) FSM-based testing methods were compared on RBAC policies specified as FSM models. The characteristics (number of resets, average test case length and test suite length) and the effectiveness of test suites generated from the W, HSI and SPY methods to five different RBAC policies were analyzed at an experiment. Later, three test prioritization methods were compared using the test suites generated in the previous investigation. A prioritization criteria based on RBAC similarity was introduced and compared to random prioritization and simple similarity. The obtained results pointed out that the SPY method outperformed W and HSI methods on RBAC domain. The RBAC similarity also achieved an Average Percentage Faults Detected (APFD) higher than the other approaches. / Controle de Acesso (CA) é um dos principais pilares da segurança da informação. Em resumo, CA permite assegurar que somente usuários habilitados terão acesso aos recursos de um sistema, e somente o acesso necessário para a realização de uma dada tarefa será disponibilizado. Neste contexto, o controle de acesso baseado em papel (do inglês, Role Based Access Control - RBAC) tem se estabelecido como um dos mais importante paradigmas de controle de acesso. Em uma organização, usuários recebem responsabilidades por meio de cargos e papéis que eles exercem e, em sistemas RBAC, permissões são distribuídas por meio de papéis atribuídos aos usuários. Apesar da aparente simplicidade, enganos podem ocorrer no desenvolvimento de sistemas RBAC e gerar falhas ou até mesmo brechas de segurança. Dessa forma, processos de verificação e validação tornam-se necessários. Teste de CA visa identificar divergências entre a especificação e o comportamento apresentado por um mecanismo de CA. Teste Baseado em Modelos (TBM) é uma variante de teste de software que se baseia em modelos explícitos de especificação para automatizar a geração de casos testes. TBM tem sido aplicado com sucesso no teste funcional, entretanto, ainda existem lacunas de pesquisa no TBM de requisitos não funcionais, tais como controle de acesso, especialmente de critérios de teste. Nesta dissertação de mestrado, dois aspectos do TBM de RBAC são investigados: métodos de geração de teste baseados em Máquinas de Estados Finitos (MEF) para RBAC; e priorização de testes para RBAC. Inicialmente, dois métodos tradicionais de geração de teste, W e HSI, foram comparados ao método de teste mais recente, SPY, em um experimento usando políticas RBAC especificadas como MEFs. As características (número de resets, comprimento médio dos casos de teste e comprimento do conjunto de teste) e a efetividade dos conjuntos de teste gerados por cada método para cinco políticas RBAC foram analisadas. Posteriormente, três métodos de priorização de testes foram comparados usando os conjuntos de teste gerados no experimento anterior. Neste caso, um critério baseado em similaridade RBAC foi proposto e comparado com a priorização aleatória e baseada em similaridade simples. Os resultados obtidos mostraram que o método SPY conseguiu superar os métodos W e HSI no teste de sistemas RBAC. A similaridade RBAC também alcançou uma detecção de defeitos superior.

Software Product Line Engineering (SPLE) is an approach used in the development of similar products, which explores the systematic reuse of software artifacts. The SPLE process has several activities executed to ensure software quality. Quality assurance is of vital importance for achieving and maintaining a high quality of all kinds of artifacts, such as products and processes. Testing activities are widely used in the industry for quality management. However, the effort for applying testing is usually high, and increasing the testing efficiency is a major concern of all systems engineering activities. A common means of increasing efficiency is automation of the test execution and the test design. Automated test design can be performed using approaches such as Model-Based Testing (MBT) in which the real behavior of a software system is compared to an abstract test model. Several techniques, processes, and strategies were developed for SPLE testing, but still many problems are open in this area of research. The challenge in focus is the reduction of the overall test effort required to test SPLE products. Test effort can be reduced by maximizing test reuse using models that take advantage of the similarity between products. The thesis goal is to automate the generation of small test-suites with high fault detection and low test redundancy between products. To achieve the goal, equivalent tests are identified for a set of products using complete and configurable test-suites. Two research directions are explored, one is product-based centered, and the other is product line-centered. For test design, test-suites that have full fault coverage were generated from state machines with and without feature constraints. A prototype implementation tool was developed for test design automation. In addition, the proposed approach was evaluated using examples, experimental studies, and an industrial case study for the automotive domain. The results indicates test effort reduction of 36% in the first research direction for a product line with 24 products, and in the second research direction increasing test effort reduction based on the number of products that require testing. For 6 products 15% reduction (from case study), and for 20 random products 50% reduction (from experimental studies). / Engenharia de Linha de Produto de Software (SPLE) é uma abordagem utilizada no desenvolvimento de produtos similares, que explora a reutilização sistemática de artefatos de software. O processo da SPLE executa várias atividades para garantir a qualidade do software. Atividades de garantia de qualidade são fundamentais para alcançar e manter altos níveis de qualidade em todos os tipos de artefatos de software, tais como produtos e processos. Atividades de teste são amplamente utilizadas na indústria para o gerenciamento de qualidade. No entanto, o esforço para a aplicação de testes geralmente é alto e melhorar a eficiência dos testes é um desafio relacionado a todas as atividades da engenharia de sistemas. Uma maneira de melhorar a eficiência da atividade de teste é automatizar a geração e execução dos testes. A geração automática de testes pode ser realizada por abordagens tais como o Teste Baseado em Modelos (TBM), em que o comportamento real do sistema de software é comparado a um modelo de teste abstrato. Várias técnicas, processos e estratégias foram desenvolvidas para o teste de SPLE, contudo, existem diversos desafios nessa área de pesquisa. O desafio em foco é a redução do esforço geral de teste necessário para testar produtos da SPLE. O esforço de teste pode ser reduzido maximizando o reuso de teste usando modelos que representam variabilidades entre os produtos. O objetivo da tese é automatizar a geração de compactos conjuntos de testes com alta capacidade de detecção de falhas e baixa redundância de teste entre produtos. Para alcançar tal objetivo, testes equivalentes são identificados para um conjunto de produtos usando conjuntos de teste completos e configuráveis. Duas direções de pesquisa são exploradas, uma centrada no produto e a outra centrada na linha de produto. Foram gerados conjuntos de teste que tenham cobertura de falhas completa a partir de máquinas de estado com e sem restrições de características. A implementação de uma ferramenta foi desenvolvida para automatizar a geração de teste. Além disso, a abordagem proposta foi avaliada usando exemplos, estudos experimentais e um estudo de caso industrial. Os resultados indicam uma redução de esforço de teste de 36% na primeira direção de pesquisa para uma linha com 24 produtos, e na segunda linha de pesquisa uma redução incremental com mais produtos a serem testados. Para 6 produtos uma redução de 15% (do estudo de caso), e para 20 produtos randomicos uma redução de 50% (dos estudos experimentais).

Multi-agent systems (MASs) can be used to model phenomena that can be decomposed into several interacting agents which exist within an environment. In particular, they can be used to model human and animal societies, for the purpose of analysing their properties by computational means. This thesis is concerned with the automated analysis of a particular kind of such social models, namely, those based on behaviourist principles, which contrasts with the more dominant cognitive approaches found in the MAS literature. The hallmark of behaviourist theories is the emphasis on the definition of behaviour in terms of the interaction between agents and their environment. In this manner, not merely re exive actions, but also learning, drives, and emotions can be defined. More specifically, in this thesis we introduce a formal agent architecture (specified with the Z Notation) based on the Behaviour Analysis theory of B. F. Skinner, and provide a suitable formal notion of environment (based on the pi-calculus process algebra) to bring such agents together as an MAS. Simulation is often used to analyse MASs. The techniques involved typically consist in implementing and then simulating a MAS several times to either collect statistics or see what happens through animation. However, simulations can be used in a more verification-oriented manner if one considers that they are actually explorations of large state-spaces. In this thesis we propose a novel verification technique based on this insight, which consists in simulating a MAS in a guided way in order to check whether some hypothesis about it holds or not. To this end, we leverage the prominent position that environments have in the MASs of this thesis: the formal specification of the environment of a MAS serves to compute the possible evolutions of the MAS as a transition system, thereby establishing the state-space to be investigated. In this computation, agents are taken into account by being simulated in order to determine, at each environmental state, what their actions are. Each simulation execution is a sequence of states in this state-space, which is computed on-the-fly, as the simulation progresses. The hypothesis to be investigated, in turn, is given as another transition system, called a simulation purpose, which defines the desirable and undesirable simulations (e.g., \"every time the agent does X, it will do Y later\"). It is then possible to check whether the MAS satisfies the simulation purpose according to a number of precisely defined notions of satisfiability. Algorithmically, this corresponds to building a synchronous product of these two transitions systems (i.e., the MAS\'s and the simulation purpose) on-the-fly and using it to operate a simulator. That is to say, the simulation purpose is used to guide the simulator, so that only the relevant states are actually simulated. By the end of such an algorithm, it delivers either a conclusive or an inconclusive verdict. If conclusive, it becomes known whether the MAS satisfies the simulation purpose with respect to the observations made during simulations. If inconclusive, it is possible to perform some adjustments and try again. In summary, then, in this thesis we provide four novel elements: (i) an agent architecture; (ii) a formal specification of the environment of these agents, so that they can be composed into an MAS; (iii) a structure to describe the property of interest, which we named simulation purpose; and (iv) a technique to formally analyse the resulting MAS with respect to a simulation purpose. These elements are implemented in a tool, called Formally Guided Simulator (FGS). Case studies executable in FGS are provided to illustrate the approach. / Sistemas multi-agentes (SMAs) podem ser usados para modelar fenômenos que podem ser decompostos em diversos agentes que interagem entre si dentro de um ambiente. Em particular, eles podem ser usados para modelar sociedades humanas e animais, com a finalidade de se analisar as suas propriedades computacionalmente. Esta tese trata da análise automatizada de um tipo particular de tais modelos sociais, a saber, aqueles baseados em princípios behavioristas, o que contrasta com as abordagens cognitivas mais dominante na literatura de SMAs. A principal característica das teorias behaviorista é a ênfase na descrição do comportamento em termos da interação entre agentes e seu ambiente. Desta forma, não apenas ações refl exivas, mas também de aprendizado, motivações, e as emoções podem ser definidas. Mais especificamente, nesta tese apresentamos uma arquitetura de agentes formal (especificada através da Notação Z) baseada na teoria da Análise do Comportamento de B. F. Skinner, e fornecemos uma noção adequada e formal de ambiente (com base na álgebra de processos pi-calculus) para colocar tais agentes juntos em um SMA. Simulações são freqüentemente utilizadas para se analisar SMAs. As técnicas envolvidas tipicamente consistem em simular um SMA diversas vezes, seja para coletar estatísticas, seja para observar o que acontece através de animações. Contudo, simulações podem ser usadas de forma a pertmitir a realização de verificações automatizadas do SMA caso sejam entendidas como explorações de grandes espaços-de-estados. Nesta tese propomos uma técnica de verificação baseada nessa observação, que consiste em simular um SMA de uma forma guiada, a fim de se determinar se uma dada hipótese sobre ele é verdadeira ou não. Para tal fim, tiramos proveito da importância que os ambientes têm nesta tese: a especificação formal do ambiente de um SMA serve para calcular as evoluções possíveis do SMA como um sistema de transição, estabelecendo assim o espaço-de-estados a ser investigado. Neste cálculo, os agentes são levados em conta simulando-os, a fim de determinar, em cada estado do ambiente, quais são suas ações. Cada execução da simulação é uma seqüência de estados nesse espaço-de-estados, que é calculado em tempo de execução, conforme a simulação progride. A hipótese a ser investigada, por sua vez, é dada como um outro sistema de transição, chamado propósito de simulação, o qual define as simulações desejáveis e indesejáveis (e.g., \"sempre que o agente fizer X, ele fará Y depois\"). Em seguida, é possível verificar se o SMA satisfaz o propósito de simulação de acordo com uma série de relações de satisfatibilidade precisamente definidas. Algoritmicamente, isso corresponde a construir um produto síncrono desses dois sistemas de transições (i.e., o do SMA e o do propósito de simulação) em tempo de execução e usá-lo para operar um simulador. Ou seja, o propósito de simulação é usado para guiar o simulador, de modo que somente os estados relevantes sejam efetivamente simulados. Ao terminar, um tal algoritmo pode fornecer um veredito conclusivo ou inconclusivo. Se conclusivo, descobre-se se o SMA satisfaz ou não o propósito de simulação com relação às observações feitas durante as simulações. Se inconclusivo, é possível realizar alguns ajustes e tentar novamente. em resumo, portanto, nesta tese propomos quatro novos elementos: (i) uma arquitetura de agente, (ii) uma especificação formal do ambiente desses agentes, de modo que possam ser compostos em um SMA, (iii) uma estrutura para descrever a propriedade de interesse, a qual chamamos de propósito de simulação, e (iv) uma técnica para se analisar formalmente o SMA resultante com relação a um propósito de simulação. Esses elementos estão implementados em uma ferramenta, denominada Simulador Formalmente Guiado (FGS, do inglês Formally Guided Simulator). Estudos de caso executáveis no FGS são fornecidos para ilustrar a abordagem.

A realização de atividades de teste é indispensável para a garantia da qualidade de um produto e para a identificação de defeitos, diminuindo custos de manutenção e evitando ao máximo o risco do cliente encontrar esses defeitos. Nessa linha, testes baseados em modelos têm se mostrado atrativos, pois o custo de geração de casos de testes e de correção de defeitos tende a ser menor. Devido à sua simplicidade conceitual e expressividade na descrição do comportamento de um sistema, um dos modelos mais usados e pesquisados na área de teste baseado em modelos são as Máquinas de Estados Finitos (MEFs). Por meio de MEFs e com apoio de ferramentas apropriadas, a geração de casos de testes para avaliar os comportamentos esperados de um sistema é automatizada, reduzindo tanto o custo da geração e da manutenção quanto as falhas humanas. Desta forma, a aplicabilidade de métodos de geração de casos de teste baseados em modelos no contexto de sistemas embarcados vem sendo investigada. O objetivo deste trabalho de mestrado consiste em investigar a aplicabilidade dos métodos de geração em cenários de teste reais, com foco em sistemas embarcados, identificando as difi- culdades e limitações do processo, bem como os requisitos essenciais para a adequação dos métodos de geração propostos na literatura e de ferramentas de apoio à atividade de teste. O foco principal do projeto é a implementação de mecanismos que atendam aos requisitos levantados, visando a usabilidade, segurança e portabilidade da ferramenta / Test activities are essential to ensure the quality of products and identify faults to reduce maintenance costs and avoid that the client finds these faults. In this sense, model-based tests have been proved useful, because the cost of generating test cases and fault correction tend to be smaller. Due to its conceptual simplicity and expressiveness in describing the behavior of a system, Finite State Machines (FSM) have been used and researched in the model-based testing area. FSMs, employed with the support of appropriate tools, enable the generation of test cases in an automated way to assess the expected behavior of a system, reducing both the generation and maintenance costs and human failures. Thus, the applicability of test cases generation methods based on models in the context of embedded systems should be investigated. Test cases generation methods based on FSM are designed to derive test cases from the model. In this context, this work aims to investigate the applicability of generation methods in real-world scenarios, focusing embedded systems. It should identify the difficulties and limitations of the process, as well as the essential requirements for the adequacy of generation methods proposed in the literature and tools to support the test activity. The main focus of the project is the implementation of mechanisms that meet the elicited requirements in order to provide usability, security and tool portability

L'internet des objets (IoT) est aujourd'hui un moyen d'innovation et de transformation pour de nombreuses entreprises. Les applications s'étendent à un grand nombre de domaines, tels que les villes intelligentes, les maisons intelligentes, la santé, etc. Le Groupe Gartner estime à 21 milliards le nombre d'objets connectés d'ici 2020. Le grand nombre d'objets connectés introduit des problèmes, tels que la conformité et l'interopérabilité en raison de l'hétérogénéité des protocoles de communication et de l'absence d'une norme mondialement acceptée. Le grand nombre d'utilisations introduit des problèmes de déploiement sécurisé et d'évolution du réseau des IoT pour former des infrastructures de grande taille. Cette thèse aborde la problématique de la validation de l'internet des objets pour répondre aux défis des systèmes IoT. Pour cela, nous proposons une approche utilisant la génération de tests à partir de modèles (MBT). Nous avons confronté cette approche à travers de multiples expérimentations utilisant des systèmes réels grâce à notre participation à des projets internationaux. L'effort important qui doit être fait sur les aspects du test rappelle à tout développeur de système IoT que: ne rien faire est plus cher que de faire au fur et à mesure. / The Internet of Things (IoT) is nowadays globally a mean of innovation and transformation for many companies. Applications extend to a large number of domains, such as smart cities, smart homes, healthcare, etc. The Gartner Group estimates an increase up to 21 billion connected things by 2020. The large span of "things" introduces problematic aspects, such as conformance and interoperability due to the heterogeneity of communication protocols and the lack of a globally-accepted standard. The large span of usages introduces problems regarding secure deployments and scalability of the network over large-scale infrastructures. This thesis deals with the problem of the validation of the Internet of Things to meet the challenges of IoT systems. For that, we propose an approach using the generation of tests from models (MBT). We have confronted this approach through multiple experiments using real systems thanks to our participation in international projects. The important effort which is needed to be placed on the testing aspects reminds every IoT system developer that doing nothing is more expensive later on than doing it on the go.

L'engouement du grand public pour les applications mobiles, dont le nombre ne cessede croître, a rendu les utilisateurs de plus en plus exigeants quant à la qualité de cesapplications. Seule une procédure de test efficace permet de répondre à ces exigences.Dans le contexte des applications embarquées sur téléphones mobiles, le test est unetâche coûteuse et répétitive principalement à cause du nombre important de terminauxmobiles qui sont tous différents les uns des autres.Nous proposons dans cette thèse le langage MATeL, un DSML (Domain-Specific ModelingLanguage) qui permet de d’écrire des scénarios de test spécifiques aux applicationsmobiles. Sa syntaxe abstraite, i.e. un méta modèle et des contraintes OCL, permet auconcepteur de manipuler les concepts métier du test d'applications mobiles (testeur, mobileou encore résultats attendus et résultats obtenus). Par ailleurs, il permet d'enrichirces scénarii avec des points de variabilité qui autorisent de spécifier des variations dansle test en fonction des particularités d'un mobile ou d'un ensemble de mobiles. La syntaxeconcrète de MATeL, qui est inspirée de celle des diagrammes de séquence UML,ainsi que son environnement basé sur Eclipse permettent à l'utilisateur de concevoir desscénarii relativement facilement.Grâce à une plateforme de test en ligne construite pour les besoins de notre projet,il est possible d'exécuter les scénarii sur plusieurs téléphones différents. La démarcheest illustrée dans cette thèse à travers des cas d'utilisation et des expérimentations quiont permis de vérifier et valider notre proposition. / Mobile applications have increased substantially in volume with the emergence ofsmartphones. Ensuring high quality and successful user experience is crucial to the successof such applications. Only an efficient test procedure allows developers to meet these requirements. In the context of embedded mobile applications, the test is costly and repetitive. This is mainly due to the large number of different mobile devices. In this thesis, we describe MATeL, a Domain-Specific Modeling Language (DSML) for designing test scenarios for mobile applications. Its abstract syntax, i.e. a meta model and OCL constraints, enables the test designer to manipulate mobile applications testing concepts such as tester, mobile or outcomes and results. It also enables him/her to enrich these scenarios with variability points in the spirit of Software Product-Line engineering, that can specify variations in the test according to the characteristics of one mobile or a set of mobiles. The concrete syntax of MATeL that is inspired from UML sequence diagrams and its environment based on Eclipse allow the user to easily develop scenarios. MATeL is built upon an industrial platform (a test bed) in order to be able to run scenarios on several different phones. The approach is illustrated in this thesis through use cases and experiments that led to verify and validate our contribution.

MÉTODOS de geração de casos de teste visam a gerar um conjunto de casos de teste com uma boa relação custo/benefício. Critérios de cobertura de teste definem requisitos de teste, os quais um conjunto de teste adequado deve cobrir. Métodos e critérios visam a selecionar casos de teste baseados em especificações, que podem ser descritas por meio de modelos, tais como Máquinas de Estados Finitos (MEF). Existem diversos métodos de geração e critérios de cobertura, diferindo entre si em função das propriedades exigidas da MEF, do custo dos testes gerados e da eficácia na revelação de defeitos. Apesar de pesquisas intensas na definição desses métodos e critérios, são poucas as ferramentas de apoio disponíveis assim como são poucos os relatos de aplicação em termos de custo e eficácia para a definição de estratégias de teste efetivas. Dessa forma, é necessário obter dados reais das vantagens e desvantagens dos métodos e critérios para subsidiar a tomada de decisão no processo de desenvolvimento de software no que tange às atividades de teste e validação. Este trabalho apresenta resultados de experimentos para avaliar o custo e a eficácia de aplicação dos métodos e critérios mais relevantes para subsidiar a definição de estratégias de teste em diversos contextos, como por exemplo, no desenvolvimento de protocolos e de sistemas reativos. Utiliza-se um protótipo desenvolvido a partir de uma reengenharia da ferramenta Plavis/FSM para apoiar os experimentos / TEST case generation methods aim to generate a test suite that offers an acceptable trade-off between cost and avail. Test coverage criteria define testing requirements, which an adequate test suite must fulfill. Methods and criteria help to select test case from specifications, which can be describe as models, for example Finite State Machines (FSM). There are several generation methods and coverage criteria that differ depending on the required properties of the FSM, the cost of generated tests and the effectiveness in revealing faults. In spite of intense researches in the definition of those methods and criteria, there are few available tools to apply them as well as application reports about cost and effectiveness issues to define effective test strategies. Thus, it is necessary to obtain real data of the advantages and disadvantages of the methods and criteria to provide decision-making in the software development process as far in the validation and test activities. This work aimed to lead experiments to evaluate the cost and the effetiveness in applying the most relevant methods and criteria to subsidize test strategies definition in several contexts as the communication protocol development and the reactive systems development. A prototype was developed based on reengineering of the Plavis/FSM tool to support the experiments

Die Arbeit widmet sich der Frage, ob sich die etablierte Technik des modellbasierten Testens (MBT) auf eine spezielle Art von Software-Komponentensystemen, den dynamischen Komponentensystemen (DCS), anwenden lässt. DCS bieten die besondere Eigenschaft, dass sich die Komposition der Komponenteninstanzen zur Laufzeit ändern kann, da in solchen Systemen jede Komponenteninstanz einen Lebenszyklus aufweist. Damit ist es möglich, im laufenden Betrieb einzelne Komponenten im Softwaresystem zu aktualisieren oder dem System neue hinzuzufügen. Derartige Eingriffe führen dazu, dass die von den Komponenteninstanzen bereitgestellte Funktionalität jederzeit eingeschränkt oder unverfügbar werden kann. Diese Eigenschaft der DCS macht die Entwicklung von Komponenten schwierig, da diese in ihrem potentiellen Verhalten darauf vorbereitet werden müssen, dass die von ihnen jeweils benötigte und genutzte Funktionalität nicht ständig verfügbar ist. Ziel dieser Dissertation ist es nun, einen systematischen Testansatz zu entwickeln, der es erlaubt, bereits während der Entwicklung von DCS-Komponenten Toleranzaussagen bzgl. ihrer dynamischen Verfügbarkeit treffen zu können. Untersucht wird, inwieweit bestehende MBT-Ansätze bei entsprechender Anpassung für den neuen Testansatz übernommen werden können. Durch die in der Dissertation entwickelten Ansätze sowie deren Implementierung und Anwendung in einer Fallstudie wird gezeigt, dass eine systematische Testfallgenerierung für dynamische Komponentensysteme mit Hilfe der Anwendung und Anpassung von modellbasierten Testtechnologien erreicht werden kann. / This dissertation devotes to the question whether the established technique of model based testing (MBT) can be applied to a special type of software component systems called dynamic component systems (DCSs). DCSs have the special characteristic that they support the change of component instance compositions during runtime of the system. In these systems, each component instance exhibits an own lifecycle. This makes it possible to update existing, or add new components to the system, while it is running. Such changes cause that functionality provided by the component instances may become restricted or unavailable at any time. This characteristic of DCSs makes the development of components difficult because required and used functionality is not available all the time. The goal of this dissertation is to develop a systematic testing approach which allows to test a component’s tolerance to dynamic availability during development time. We analyze, to what extend existing MBT approaches can be reused or adapted. The approaches of this dissertation has been implemented in a software prototype. This prototype has been used in a case study and it has been showed, that systematic test generation for DCSs can be done with the help of MBT.

<p>Testing is the dominant verification technique used in the software industry today. The use of automatic test case execution increases, but the creation of test cases remains manual and thus error prone and expensive. To automate generation and selection of test cases, model-based testing techniques have been suggested.</p><p>In this thesis two central problems in model-based testing are addressed: the problem of how to formally specify coverage criteria, and the problem of how to generate a test suite from a formal timed system model, such that the test suite satisfies a given coverage criterion. We use model checking techniques to explore the state-space of a model until a set of traces is found that together satisfy the coverage criterion. A key observation is that a coverage criterion can be viewed as consisting of a set of items, which we call coverage items. Each coverage item can be treated as a separate reachability problem. </p><p>Based on our view of coverage items we define a language, in the form of parameterized observer automata, to formally describe coverage criteria. We show that the language is expressive enough to describe a variety of common coverage criteria described in the literature. Two algorithms for test case generation with observer automata are presented. The first algorithm returns a trace that satisfies all coverage items with a minimum cost. We use this algorithm to generate a test suite with minimal execution time. The second algorithm explores only states that may increase the already found set of coverage items. This algorithm works well together with observer automata.</p><p>The developed techniques have been implemented in the tool CoVer. The tool has been used in a case study together with Ericsson where a WAP gateway has been tested. The case study shows that the techniques have industrial strength.</p>