Software testing is becoming more prominent within the automotive industry due to more complex systems, and functions are implemented in the vehicles. The vehicles in the future will have the functionality to manage different levels of automation, which also means that vehicles driven by humans will have more supportive functionality to increase safety and avoid accidents. These functionalities result in a massive growth in the number of test scenarios to indicate that the vehicles are safe, and this makes it impossible to continue performing the tests in the same way as it has been done until today. The new conditions require that the test scenarios and Test Cases both be generated and executed automatically. In this thesis, an investigation and evaluation are performed to analyze the Automatic Test Case Generation methods available for inputs from Natural Language Requirements in an automotive industrial context at NEVS AB. This study aims to evaluate the NAT2TEST strategy by replacing the manual method and obtain a similar or better result. A comparative analysis is performed between the manual and automated approaches for various levels of requirements. The results show that utilizing this strategy in an industrial scenario can improve efficiency if the requirements to be tested are for well-documented lower-level requirements.

This dissertation introduces a hybrid strategy for automated testing of Android applications that combines reinforcement learning and test suite prioritization. These approaches aim to improve the effectiveness of the testing process by employing reinforcement learning algorithms, namely Q-learning and SARSA (State-Action-Reward-State-Action), for automated test case generation. The studies provide compelling evidence that reinforcement learning techniques hold great potential in generating test cases that consistently achieve high code coverage; however, the generated test cases may not always be in the optimal order. In this study, novel test case prioritization methods are developed, leveraging pairwise event interactions coverage, application state coverage, and application activity coverage, so as to optimize the rates of code coverage specifically for SARSA-generated test cases. Additionally, test suite prioritization techniques are introduced based on UI element coverage, test case cost, and test case complexity to further enhance the ordering of SARSA-generated test cases. Empirical investigations demonstrate that applying the proposed test suite prioritization techniques to the test suites generated by the reinforcement learning algorithm SARSA improved the rates of code coverage over original orderings and random orderings of test cases.

The requirements for software products have increased in recent years. This is both due to more complex technology as well as more requirements from the customers. An approach to solve this issue is by using a software product line (SPL) where reusable assets are developed to produce more tailor-made products with reduced time to market. When creating reusable assets, one also wants to reuse the tests for them. In order to do that, it is important to understand what to test, and where to test. A good test strategy is thereby crucial in order to avoid testing becoming a bottleneck for efficient software development. The purpose of this thesis was to investigate how to create a good testing strategy for products in an SPL. This was done by collecting information about the current testing process in order to understand which requirements that exist and how they are addressed. The core foundation of an SPL is to utilize variation to create new products. To understand the existing variation of the product and the test cases a feature modeling and similarity analysis was done. The result from them made it possible to create a test strategy and categorization of test cases that can be used to ease the reuse of test cases for new variants of the product. The resulting test strategy presented proposes feature modeling as the basis of test creation and categorization as a tool to enable easier reuse of test cases. The results of the study indicate that using SPL theory in testing can introduce a better test strategy and test case creation process which is beneficial for the whole development process. / Kraven på mjukvaruprodukter har ökat de senaste åren. Detta beror delvis på mer komplicerad teknologi, men även på att kunderna ställer högre krav på de produkter de använder. Ett tillvägagångssätt för att lösa detta är att använda en software product line. I en software product line utvecklas återanvändningsbara delar som sedan kan kombineras och därmed skapa skräddarsydda produkter på ett mer effektivt sätt. När man skapar återanvändningsbara mjukvarudelar vill man också återanvända testerna för dem. För att göra det är det viktigt att förstå vad som ska testas och varför. En bra teststrategi är därmed avgörande för att undvika att testning blir en flaskhals för effektiv mjukvaruutveckling. Syftet med detta arbete var att undersöka hur man skapar en bra teststrategi för produkter i en software product line. Detta gjordes genom att samla in information om den nuvarande testprocessen för att förstå vilka krav som finns och hur de hanteras. En huvudaspekt i software product lineteori är att använda variation för att skapa nya produkter. För att förstå variationen i produkten och dess testfall gjordes en funktionsmodellering och likhetsanalys. Resultaten från dem gjorde det möjligt att skapa en teststrategi och en kategorisering av testfall som kan användas för att underlätta återanvändningen av testfall för nya varianter av produkten. Den resulterande teststrategin som presenteras föreslår funktionsmodellering som grund för testskapande och kategorisering som ett verktyg för att möjliggöra enklare återanvändning av testfall. Resultaten av studien tyder på att användning av software product lineteori vid testning kan resultera i en bättre teststrategi och enklare process för skapandet av nya testfall.

Modern-day vehicles consist of numerous electronic computing devices with accompanying software. Since vehicles are generally classified as safety-critical systems, rigorous testing strategies have to be deployed to ensure correct operation of the embedded software. Testing in an active sense is understood to be the main testing method for software in general. The main characteristic of active testing is that test cases have complete control over the system under test, warranting sequential execution of test cases. To complement active testing a method called passive testing is being researched, where the main benefit compared to active testing is that the evaluation of test cases becomes parallelizable. As passive testing opens up for concurrency, the issue of not recognizing faulty behavior arises. This could be the case when simultaneously testing functions which share an output, and the incorrect behavior of a function is being masked by the correct behavior of another function. To avoid vacuous test results, the evaluation of dependent test cases can be separated. Previous work describes the process of extracting dependencies between test cases from requirement specifications. However, this approach is not suitable when test cases are derived from various artifacts. In this thesis we present a syntax-based approach for dependency discovery between test cases. The approach was evaluated through a case study using integration test cases developed for active hardware-in-the-loop testing at Scania CV. Dependencies between 946 test cases have been extracted and the test cases were grouped into three categories, with 286 test cases being identified as independent. The approach shows some potential for evaluating test cases written for active testing, but the results indicate that using test cases written specifically for passive testing may yield better results. The approach also proved to be useful for finding bugs and deviations in scripts. Thus, the method has proven useful for discovering dependencies from syntax, but the results are strongly dependent on the complexity and commonality of the scripts. / Moderna fordon innehåller åtskilliga elektroniska kontrollenheter med medföljande programvara. Eftersom fordon ofta anses vara säkerhetskritiska system krävs robusta teststrategier av den inbyggda mjukvaran för att säkerställa korrekt drift. Aktiv testning är för nuvarande den vanligaste metoden för att testa mjukvara. Den huvudsakliga egenskapen för aktiv testning är att testfall har total kontroll över systemet som är satt under test, vilket medför sekventiell utvärdering av testfall. För att komplettera aktiv testning undersöks en metod som kallas passiv testning, där den största fördelen gentemot aktiv testning är att utvärderingen av testfall kan ske parallellt. Att utvärdera testfall samtidigt medför dock att felaktigt beteende i mjukvaran potentiellt kan maskeras. Detta kan ske då flera funktioner delar utsignaler, och en funktions felaktiga beteende maskeras av en annan funktions korrekta beteende. För att undvika tomma sanningar som testresultat, kan utvärderingen av testfall som har ett beroende mellan dem separeras. Föregående arbeten inom området beskriver processen för att extrahera beroenden mellan testfall från kravspecifikationer. Denna metod är dock inte lämplig när testfall är baserade på flera olika artefakter. I detta examensarbete presenterar vi en syntaxbaserad metod för att hitta beroenden mellan testfall. Metoden utvärderades genom en fallstudie på Scania CV, där aktiva integrationstestfall för hardware-in-the-loop användes för analys. Beroenden mellan 946 testfall kunde extraheras och testfallen grupperades i tre olika kategorier, där 286 klassificerades som oberoende. Metoden visar potential för att analysera testfall skrivna för aktiv testning, men resultaten indikerar att testfall skrivna specifikt för passiv testning kan generera ett bättre resultat. Ett oväntat men positivt utfall är att metoden i viss grad kan användas för att hitta buggar och avvikelser i skript. Således har metoden visats användbar för att hitta beroenden från syntax, men resultatet beror starkt på skriptens komplexitet och gemensamhet.

The object of this study is the processes of testing web applications. The subject of the study is: automation of web application testing processes through the introduction of artificial intelligence. The purpose of the work is to develop a testing automation system for web applications, which will increase the speed of task execution and the adoption of management decisions related to the quality of the final product. Research methods - literature review, analysis of modern testing automation methods, machine learning. The result of the work is a system for automatic generation of test scenarios. / Объектом данного исследования является процессы тестирования web-приложений. Предметом исследования является: автоматизация процессов тестирования web приложений за счет внедрения искусственного интеллекта. Цель работы – разработать систему автоматизации тестирования для web- приложений, за счет которой будет повышена скорость выполнения задач и принятие управленческих решений, связанных с качеством конечного продукта. Методы исследования – литературный обзор, анализ современных методов автоматизации тестирования, машинное обучение. Результатом работы является система для автоматической генерации тестовых сценариев.

<p>In this research paper, it evaluates the benefits achievable from static testing framework by analyzing and transforming the <em>JUnit3.8 </em>source code and static execution of transformed code. Static structure enables us to analyze the code statically during creation and execution of test cases. The concept of research is by now well established in static analysis and testing development. The research approach is also increasingly affecting the static testing process and such research oriented work has proved particularly valuable for those of us who want to understand the reflective behavior of <em>JUnit3.8 Framework</em>.</p><p><em> JUnit3.8 Framework</em> uses <em>Java Reflection API</em> to invoke core functionality (test cases creation and execution) dynamically. However, <em>Java Reflection API</em> allows developers to access and modify structure and behavior of a program.  Reflection provides flexible solution for creating test cases and controlling the execution of test cases. Java reflection helps to encapsulate test cases in a single object representing the test suite. It also helps to associate each test method with a test object. Where reflection is a powerful tool to perform potential operations, on the other hand, it limits static analysis. Static analysis tools often cannot work effectively with reflection.</p><p>In order to avoid the reflection, <em>Static Testing Framework</em> provides a static platform to analyze the <em>JUnit3.8</em> source code and transform it into non-reflective version that emulates the dynamic behavior of <em>JUnit3.8</em>. The transformed source code has possible leverage to replace reflection with static code and does same things in an execution environment of <em>Static Testing Framework</em> that reflection does in <em>JUnit3.8</em>. More besides, the transformed code also enables execution environment of <em>Static Testing Framework</em> to run test methods statically. In order to measure the degree of efficiency, the implemented tool is evaluated. The evaluation of <em>Static Testing Framework</em> draws results for different Java projects and these statistical data is compared with <em>JUnit3.8</em> results to measure the effectiveness of <em>Static Testing Framework</em>. As a result of evaluation, <em>STF</em> can be used for static creation and execution of test cases up to <em>JUnit3.8</em> where test cases are not creating within a test class and where real definition of constructors is not required. These problems can be dealt as future work by introducing a middle layer to execute test fixtures for each test method and by generating test classes as per real definition of constructors.</p>

In this research paper, it evaluates the benefits achievable from static testing framework by analyzing and transforming the JUnit3.8 source code and static execution of transformed code. Static structure enables us to analyze the code statically during creation and execution of test cases. The concept of research is by now well established in static analysis and testing development. The research approach is also increasingly affecting the static testing process and such research oriented work has proved particularly valuable for those of us who want to understand the reflective behavior of JUnit3.8 Framework. JUnit3.8 Framework uses Java Reflection API to invoke core functionality (test cases creation and execution) dynamically. However, Java Reflection API allows developers to access and modify structure and behavior of a program.  Reflection provides flexible solution for creating test cases and controlling the execution of test cases. Java reflection helps to encapsulate test cases in a single object representing the test suite. It also helps to associate each test method with a test object. Where reflection is a powerful tool to perform potential operations, on the other hand, it limits static analysis. Static analysis tools often cannot work effectively with reflection. In order to avoid the reflection, Static Testing Framework provides a static platform to analyze the JUnit3.8 source code and transform it into non-reflective version that emulates the dynamic behavior of JUnit3.8. The transformed source code has possible leverage to replace reflection with static code and does same things in an execution environment of Static Testing Framework that reflection does in JUnit3.8. More besides, the transformed code also enables execution environment of Static Testing Framework to run test methods statically. In order to measure the degree of efficiency, the implemented tool is evaluated. The evaluation of Static Testing Framework draws results for different Java projects and these statistical data is compared with JUnit3.8 results to measure the effectiveness of Static Testing Framework. As a result of evaluation, STF can be used for static creation and execution of test cases up to JUnit3.8 where test cases are not creating within a test class and where real definition of constructors is not required. These problems can be dealt as future work by introducing a middle layer to execute test fixtures for each test method and by generating test classes as per real definition of constructors.

The software industry is expanding at a rapid rate. To keep up with the fast-growing and ever-changing technologies, it has become necessary to produce high-quality software in a short time and at an affordable cost. This research aims to demonstrate to Java developers the use of Automated Test Case Generation (ATCG) tools by presenting the tools that exist today, their usefulness, and their effectiveness. The main focus is on the automated testing tools for the Java industry, which can help developers achieve their goals faster and make better software. Moreover, the discussion covers the availability, features, prerequisites, effectiveness, and limitations of the automated testing tools. Among these tools, the most widely used are Evosuite, JUnit, TestNG, and Selenium. Each tool has its advantages and purpose. Furthermore, these ATCG-tools were compared to provide a clear picture to Java developers, answer the research questions, and show strengths and limitations of each selected tool. Results show that there is no single ultimate tool that can do all kinds of testing independently. It all depends on what the developer aims to achieve. If one tool is good at generating unit test cases for Java classes, another tool is good at testing the code security through penetration testing. Therefore, the Java developers may choose a tool/s based on their requirements. This study has revealed captivating findings regarding the ATCG-tools, which ought to be explored in the future.

Nous proposons dans cette thèse les approches de test pour la composition de services web. Nous nous intéressons aux test unitaire et d’intégration d’une orchestration de services web. L’aspect de vérification d’exécution en-ligne est aussi consideré. Nous définissons une plateforme de test unitaire pour l’orchestration de services web qui compose une architecture de test, une relation de conformité et deux approches de test basés sur le modèle de machine à l’états finis étendues temporisés: l’approche offline où les activités de test comme la génération de cas de test temporisé, l’exécution de test et l’assignement de verdict sont appliquées en séquentielle tandis que ces activités sont appliquées en parallèle dans l’approche online. Pour le test d’intégration d’une orchestration, nous combinons deux approches: active et passive.Au debut, l’approche active est utilisée pour activer une nouvelle session d’orchestration par l’envoi d’un message de requête SOAP. Après, tous les messages d’entré et de sortie de l’orchestration sont collectés et analysés par l’approche passive.Pour l’aspect de vérification d’exécution en-ligne, nous nous intéressons à la vérification d’une trace qui respecte un ensemble des constraintes, noté règles, ou pas. Nous avons proposé extendre le langage Nomad en définissant des constraintes sur chaque action atomique et un ensemble de corrélation de données entre les actions pour définir des règles pour le service web. Ce langage nous permet de définir des règles avec le temps futur et passé, et d’utiliser des opérations NOT, AND, OR pour combiner quelque conditions dans le contexte de la règle. Ensuite, nous proposons un algorithme pour vérifier l’exactitude d’une séquence des messages en parallèle avec le moteur de collecte de trace. / In this thesis, we propose the testing approaches for web service composition. We focus on unit, integrated testing of an orchestration of web services and also the runtime verification aspect. We defined an unit testing framework for an orchestration that is composed of a test architecture, a conformance relation and two proposed testing approaches based on Timed Extended Finite State Machine (TEFSM) model: offline which test activities as timed test case generation, test execution and verdict assignment are applied in sequential, and online which test activities are applied in parallel. For integrated testing of an orchestration, we combines of two approaches: active and passive. Firstly, active approach is used to start a new session of the orchestration by sending a SOAP request. Then all communicating messages among services are collected and analyzed by a passive approach. On the runtime verification aspect, we are interested in the correctness of an execution trace with a set of defined constraints, called rules. We have proposed to extend the Nomad language, by defining the constraints on each atomic action (fixed conditions) and a set of data correlations between the actions to define the rules for web services. This language allows us to define a rule with future and past time, and to use the operations: NOT, AND, OR to combines some conditions into a context of the rule. Afterwards, we proposed an algorithm to check correctness of a message sequence in parallel with the trace collection engine. Specifically, this algorithm verifies message by message without storing them.

A Arquitetura Orientada a Serviços (SOA) é um estilo arquitetural formado por um conjunto de restrições que visa promover a escalabilidade e a flexibilidade de um sistema, provendo suas funcionalidades como serviços. Nos últimos anos, um estilo alternativo foi proposto e amplamente adotado, que projeta as funcionalidades de um sistema como recursos. Este estilo arquitetural orientado a recursos é chamado de REST. O teste de serviços web em geral apresenta vários desafios devido a sua natureza distribuída, canal de comunicação pouco confiável, baixo acoplamento e a falta de uma interface de usuário. O teste de serviços RESTful (serviços que utilizam o REST) compartilham estes mesmos desafios e ainda necessitam que suas restrições sejam obedecidas. Estes desafios demandam testes mais sistemáticos e formais. Neste contexto, o teste baseado em modelos (TBM) se apresenta como um processo viável para abordar essas necessidades. O modelo que representa o sistema deve ser simples e ao mesmo tempo preciso para que sejam gerados casos de teste com qualidade. Com base nesse contexto, este projeto de mestrado propõe uma abordagem baseada em modelos para testar serviços RESTful. O modelo comportamental adotado foi a máquina de estados de protocolos UML, capaz de formalizar a interface do serviço enquanto esconde o seu funcionamento interno. Uma ferramenta foi desenvolvida para gerar automaticamente os casos de teste usando critérios de cobertura de estados e transições para percorrer o modelo / Service Oriented Architecture (SOA) is an architectural style consisting of a set of restrictions aimed at promoting the scalability and flexibility of a system, providing its functionalities as services. In recent years, an alternative style was proposed and widely adopted, which designs the system\'s functionalities as resources. This resource oriented architectural style is called REST. In general, the test of web services has several challenges due to its distributed nature, unreliable communication channel, low coupling and the lack of a user interface. Testing RESTful web services (services that use REST) share these same challenges and also need to obey the REST constraints. These challenges require a more systematic and formal testing approach. In this context, model based testing presents itself as a viable process for addressing those needs. The model that represents the system should be simple and precise enough to generate quality test cases. Based on this context, this work proposes a model based approach to test RESTful web services. The behavioral model used was the UML protocol state machine, which is capable to provide a formalization of the service interface, while hiding its internal behaviour. A tool was developed to automatically generate test cases using the state and transition coverage criteria to traverse the model