Graphical user interfaces represent significant part of computer systems. This causes an increase of demands for their proper behavior, which can be achieved by a proper testing during development. This thesis describes the principles of model based testing for automated generation of test cases. In this way, the productivity of test teams can increase. The thesis defines model for modeling the most common GUI elements, algorithm for obtaining such a model from the GUI, and a tool for definition of coverage criteria.

Traceability in the software industry is a topic which has been studied for a long time, but there still remain questions to be answered. This thesis looks into the state of software traceability in the software industry is today. It is also a case study in implementing traceability in a model-based testing process at a large automotive company. The thesis finds that while progress has been made in identifying the important aspects of traceability, there does not yet exist a clear consensus on general traceability models which can be used to implement traceability in an arbitrary company or project. Through a case study in implementing traceability in a model-based testing process, the thesis gives one example of how this could potentially be done.

Model-based testing (MBT) is using a model of expected behavior of the system to automatically generate a set of tests. It aims at reducing the testing cost when compared to the traditional testing techniques. This work focuses on testing a real-world software system using the selected MBT tool OSMO. The tested system is responsible for compiling business rules and it is one of the main components of the Drools platform, developed by Red Hat. The work describes the introduction of MBT considering the good reception from the community of developers, then the creation of compiler input models and evaluation of the newly created test suite. The usage of the MBT resulted in detection of five reported and three potential issues in the tested code. Using the Drools compiler example, the work summarizes the main strengths and also weaknesses of practical use of MBT techniques.

In Model-Based Testing (MBT) the main goal is to test a system by designing models which describe the functionality of the system to test. Subsequently, test cases are obtained from the model, and these test cases can be executed automatically. Experience has shown that the learning curve for learning MBT can be steep - especially for people who do not have previous programming experience. This is because the language used to design models uses programming language concepts. In this thesis we describe a tool which automatically generates models, given an initial set of requirements. The advantage of this tool is that users do not need to learn a model-based testing language to design models, but instead they must learn to use a high-level of abstraction and a Graphical User Interface to specify their test cases. We demonstrate the value of the tool by using it to design models that generate test cases for telecommunications system, but show that this tool can be adapted for use in testing similar systems. The application of this tool can facilitate traditional phase-based software development methods, by saving a considerable amount of time and resources. In addition, when applied to agile software development, the reduced time required for testing because of the use of our tool helps shortening the feedback loops between designing and testing, thus increasing team efficiency within every iteration. / I Model-Based Testing (MBT) är det huvudsakliga målet att testa ett system genom modeller som beskriver systemets unktionalitet för att testa. Därefter erhålls testfall från modellen, och dessa testfall kan utföras automatiskt. Erfarenheten har visat att inlärningskurvan för lärande MBT kan vara branta - särskilt för personer som inte har tidigare erfarenhet av programmering. Detta beror på det språk som används för användning programmeringsspråk begrepp. I denna avhandling beskriver vi ett verktyg som automatiskt genererar modellerna, med tanke på en första uppsättning krav. Fördelen med detta verktyg är att användarna inte behöver lära sig en modellbaserad testning av språket att konstruera modeller, utan de måste lära sig att använda en hög nivåav abstraktion och ett grafiskt användargränssnitt för att ange sina testfall. Vi demonstrerar värdet av verktyget genom att använda den för att konstruera modeller som genererar testfall för telekommunikationssystem, men visar att detta verktyg kan anpassas för användning vid testning av liknande system. Tillämpningen av detta verktyg kan underlätta traditionella fas-baserade metoder mjukvaruutveckling, genom att spara en avsevärd tid och resurser. Dessutom, när det tillämpas på Agile Software utveckling, minskade tid som krävs för att testa på grund av användningen av vårt verktyg hjälper förkorta återkopplingar mellan design och testning, vilket ökar teamet effektiviteten inom varje iteration. / En Model-Based Testing (MBT), el objetivo principal es testear un sistema mediante el diseño de modelos que describan su funcionalidad. En consecuencia, estos modelos generan test cases que pueden ser ejecutados automáticamente en dicho sistema. La experiencia nos muestra que la curva de aprendizaje en el caso de MBT puede ser pronunciada, especialmente para aquellos sin ninguna experiencia previa en programación. Esto se debe a que los lenguajes usados para diseñar modelos usan conceptos intrínsecos a los lenguajes de programación. En este Proyecto Fin de Carrera, describimos una herramienta que genera automáticamente modelos, dado un conjunto de requisitos inicial. La ventaja que ofrece esta herramienta es que los usuarios no requieren el aprendizaje de ninguno lenguaje de modelado a la hora de diseñar modelos, sino que tan solo deben aprender a utilizar una Interfaz de Usuario Gráfica (GUI), a un alto nivel de abstracción, para especificar sus test cases. Demostramos el valor de esta herramienta mediante su aplicación en un nuevo sistema de telecomunicaciones en fase de pruebas de Ericsson, mostrando al mismo tiempo que puede ser utilizada en el testeo de sistemas similares. La aplicación de esta herramienta puede facilitar los métodos de desarrollo de software tradicionales mediante el ahorro de una cantidad considerable de tiempo y recursos. Además, aplicada a métodos de desarrollo ágil de software, el tiempo reducido requerido para el testing a causa del uso de esta herramienta ayuda a acortar los plazos entre diseño y testing, y en consecuencia, incrementando la eficiencia del equipo en cada iteración.

Akzeptanztests werden in Scrum-Projekten durchgeführt, um umgesetzte Anforderungen iterativ und inkrementell gegen die Erwartungen des Product Owners zu prüfen. Dazu müssen Testartefakte kontinuierlich spezifiziert, gewartet und ausgeführt werden. Im Projektverlauf werden die Durchführung und Wartung des Testprozesses zunehmend zu zeit- und ressourcenkritischen Aufgaben, die mit einer manuellen Testdurchführung nicht mehr zu bewerkstelligen sind. Es wird ein ausgereifter Testautomatisierungsansatz benötigt. In dieser Arbeit wurde ein Testverfahren entworfen, welches die Vorteile des Model-Based Testing (MBT) mit anderen Test-First-Ansätzen, wie dem Behavior-Driven Development und Acceptance Test-Driven Development kombiniert, um die automatisierte Durchführung von Akzeptanztests im Scrum-Vorgehen zu integrieren. Dafür wurde untersucht, wie Anforderungen in Form von User Storys für die Testmodellentwicklung formalisiert werden können, wie die Rückverfolgbarkeit vom Testfall zur Anforderung und zurück gewährleistet werden kann und wie abstrakte Testfälle für die automatische Testdurchführung konkretisiert werden können. Zudem wurden ein Automatisierungs- und Durchführungsansatz für die Nutzung von GraphWalker mit Selenium als Adapterschicht zur Graphical User Interface (GUI)-Testautomatisierung sowie Möglichkeiten der Nutzung von Zustandsmodellen zur Modellierung fachlich oder logisch unerwünschter Pfade dargelegt. Das Testverfahren wurde prototypisch bei einem Fallbeispiel implementiert, um konkrete Anwendungsfälle umzusetzen und Rückschlüsse aus der praktischen Anwendung ziehen zu können. Die Ergebnisse der Arbeit zeigen, dass sich das konzipierte Testverfahren für die Automatisierung von Akzeptanztests eignet und das Vorgehen sowohl von Forschenden als auch Praktizierenden zur Implementierung eines MBT-Verfahrens verwendet werden kann.

Les systèmes réactifs sont modélisés avec différents types d'automates, tels que les systèmes de transitions symboliques à entrée sortie (IOSTS). L'exécution symbolique d'un IOSTS permet la génération de cas de test qui peuvent être exécutés sur une implantation concrète, afin de déterminer si elle est conforme à son modèle. Dans ce document, nous étendons les IOSTS avec des appels de fonctions utilisateur et analysons leur impact sur le système entier et viceversa. Cette thèse comble l'écart entre une approche basée sur le modèle où les fonctions utilisateur sont abstraites et une approche basée sur le code où les petits morceaux de code sont considérés séparément, indépendamment de la façon dont ils sont combinés. Selon le niveau de connaissance que nous avons sur ces fonctions, elles sont modélisées soit par une spécification complète, soit par une spécification partielle, soit juste comme des boîtes noires fournies sans aucune connaissance. Premièrement, lorsque les fonctions sont partiellement connues, nous utilisons des bouchons définis par des tables contenant des tuples représentatifs des données d'entrée/sortie. L'approche proposée emprunte au test "concolic", l'idée de mélanger l'exécution symbolique avec l'information obtenue à partir d'exécutions concrètes des fonctions (tables). Deuxièmement, si l'utilisateur est prêt à fournir d'autres spécifications, il serait intéressant d'utiliser des représentations plus complètes pour les fonctions. Par conséquent, nous proposons d'abstraire des comportements des fonctions par des contrats pré/post. Ensuite, nous étendons l'exécution symbolique en analysant les fonctions via leurs contrats. Enfin, lorsque les fonctions appelées sont complètement inconnues, nous présentons une approche pour extraire de nouveaux contrats pour eux en explorant les contraintes issues de l'exécution symbolique de l'IOSTS. De tels contrats reflètent les contraintes des fonctions qui rendent possible un certain comportement (exigence). / Reactive systems are modeled with various kinds of automata, such as Input Output Symbolic Transition Systems (IOSTS). Symbolic execution over an IOSTS allows test cases generation that can be executed on a concrete implementation, in order to determine whether it is conforming to its model. In this dissertation, we aim at extending the IOSTS framework with explicit user-defined function calls and analyze their impact on a whole system and vice-versa. The thesis bridges the gap between a model-based approach in which user-defined functions are abstracted away and a code-based approach in which small pieces of code are considered separately regardless of the way they are combined. According to the level of knowledge we have about these functions, they are modeled either by a complete specification, a partial specification, or even just as black-boxes provided without any knowledge. First, when functions are partially known, we use function stubs defined by tables containing representative input/output data tuples. The proposed approach borrows from concolic testing, the idea of mixing symbolic execution with information obtained from instrumented concrete executions (function tables). Second, if the user is willing to provide further specifications, it would be interesting to use more complete representations for called functions. Hence, we propose to abstract function behaviors by means of pre/post contracts. Then we extend symbolic execution by analyzing the functions through their contracts. Finally, when called functions are completely unknown, we present an approach to extract new contracts for them by exploring constraints coming from the IOSTS symbolic execution. Such contracts reflect constraints on the functions that make some behavior (requirement) feasible.

The automatic creation of test cases has been a well-researched area in recent years. Indeed, the industry’s testing procedure still uses the traditional way of manual practices. However, investigations are continued to deliver new methods, but research results have not been fully adopted. In this paper, the investigated method applies the model-based testing (MBT) method to evaluate the ability to automate the creation of hardware-in-the-loop (HIL) test cases, where generated tests are created using MBT tools. The tools’ properties were compared to each other through a literature study, with the result of selecting tools to be used in a case study, and the tools selected were GraphWalker and MoMuT. The generated test cases perform similarly to their manual counterparts regarding how the test cases achieved full requirements coverage. When comparing the effort needed for applying the methods, a comparable effort is required for creating the first iteration, while with every subsequent update, MBT will require less effort compared to the manual process. Both methods achieve 100% requirements coverage, and since manual testing is created and executed by humans, some requirements are favoured over others due to company demands, while MBT tests will be generated randomly.In addition, a comparison between the used tools showcased the differences in the models’ design and their test case generation. The comparison showed that GraphWalker has a more straightforward design method and is better suited for smaller systems, while’s MoMuT can handle more complex systems but has a more involved design method.The results of the thesis showed that using MBT tools proved helpful as it covers the system requirements, can be executed in HIL and helps discover faults within the requirements and HIL system. These facts satisfy the companies’ demands. This thesis shows a promising improvement in automating the test process within the vehicular domain.

The advent of Software Defined Networking (SDN) and programmable data planes has revolutionized the networking domain, enabling the programming of networking functions down to the silicon level responsible for data packet switching. Unfortunately, while this programmability offers greater flexibility and control, it also increases the likelihood of introducing software bugs. To counter this risk, rigorous testing methodologies and strategies are essential to ensure the reliability, security, and stability of SDN deployments. A comprehensive approach should combine various techniques, including formal verification, fuzz, and performance testing. Model-Based Testing (MBT) is a technique that can significantly enhance the effectiveness of SDN testing. By leveraging formal models of the system under test, MBT automatically generates test cases that can help identify potential issues in network configuration, data plane programming, and network protocols. Utilizing MBT allows network administrators to systematically explore SDN components’ possible states and transitions, resulting in a higher level of coverage and confidence in the system’s overall stability and security. However, a lack of information on applying MBT in an SDN environment challenges its full implementation and utilization in this field. This master thesis aims to investigate and demonstrate the application of MBT to programmable data plane functions. This work uses VLAN tagging as the target data plane function, and AltWalker is employed as the MBT tool for generating and executing tests on an SDN switch. The results present an initial testing methodology that, when applied to the VLAN tagging function, can provide insights into the potential benefits and challenges of using MBT for SDN testing. This thesis lays the groundwork for further exploration and refinement of MBT methodologies in the context of SDN and programmable data plane functions.

yes / Engineering design in general and system design of embedded software have a direct impact on the final engineering product and the software implementation, respectively. Guaranteeing that the models utilised meet the specified requirements is beneficial in detecting misbehaviour and software flaws. This requires an integrated approach, combining verification and model-based testing methodology and notations and methods from system engineering and software engineering. In this paper, we propose a model-based approach integrating various notations utilised in the functional design of complex systems with formal verification and testing. We illustrate our approach on the cruise control system of an e-Bike case study.

As técnicas de teste podem ser divididas, num primeiro nível, naquelas baseadas no código (caixa branca) e naquelas baseadas na especificação (caixa preta ou funcionais). Nenhuma delas é completa pois visam a identificar tipos diferentes de defeitos e a sua utilização em conjunto pode elevar o nível de confiabilidade das aplicações. Assim, tornam-se importantes estudos que contribuam para um melhor entendimento da relação existente entre técnicas funcionais e estruturais, como elas se complementam e como podem ser utilizadas em conjunto. Este trabalho está inserido no contexto do projeto PLAVIS (Plataforma para Validação e Integração de Software em Aplicações Espaciais), e tem como objetivo realizar um estudo comparativo entre as técnicas de geração de casos de teste funcionais (baseadas nas especificações formais) e os critérios estruturais baseados em fluxo de controle e fluxo de dados, aplicados nas implementações. Num contexto específico, esse estudo deve fornecer dados de como se relacionam essas duas técnicas (funcional e estrutural) gerando subsídios para sua utilização em conjunto. Num contexto mais amplo - o do projeto PLAVIS - visa a estabelecer uma estratégia de teste baseada em critérios funcionais e estruturais e que possam, juntamente com as ferramentas que dão suporte a eles, compor um ambiente de teste disponível à utilização em aplicações espaciais dentro do INPE / Testing techniques can be divided, in high level, in code-based ones (white box) and specification based ones (black box). None of them are complete as they intend to identify different kinds of faults. The use of them together can increase the application confidence level. Thus, it is important to investigate the relationship between structural testing techniques and functional testing techniques, how they complete themselves and how they can be used together. This paper was developed in the context of the Plavis (PLAtform of software Validation & Integration on Space systems) project. This project provides comparative studies between functional generation testing techniques (based on formal specifications) and structural generation testing techniques, such as control-flow and data-flow criteria, applied in the implementation. In a specific context, this study provides data about the relationship between these techniques and how they can be used together. In the context of the Plavis project, the goal is to provide a testing strategy, based on functional and structural criteria, and a set of tools, composing a testing environment to be used in Space Applications projects, at INPE