This thesis is about software testing and mainly focused on performance testing of web applications. The introductory chapters outline problems of testing, identification of key issues, explanation of general concepts and software quality, differences between desktop and web application testing and finally - performance testing. Then, there is chosen a web application suitable for testing (Kentico CMS 6) and comparison of existing tools for performance and load testing. One of them (Microsoft Visual Studio 2010 Ultimate Edition) is chosen for the further testing. Therefore, several test scenarios are designed and implemented (including demonstration of creating, editing and debugging, extending with plug-ins, maintaining, running in a distributed environment, etc.). Finally, there are introduced testing reports and suggestions for the further testing.

First, a few bugs from a list of common bug were chosen and patterns describing these bugs were inferred. Then, detectors searching for such patterns were implemented as plug-ins to FindBugs static analyzer. Finally, detectors were used to detect bugs in development version of JBoss AS. Results are presented at the end of this paper.

With the extreme growth in third party cloud applications, increased exposure of applications to the internet, and the impact of successful breaches, improving the security of software being produced is imperative. Static analysis tools can alert to quality and security vulnerabilities of an application; however, they present developers and analysts with a high rate of false positives and unactionable alerts. This problem may lead to the loss of confidence in the scanning tools, possibly resulting in the tools not being used. The discontinued use of these tools may increase the likelihood of insecure software being released into production. Insecure software can be successfully attacked resulting in the compromise of one or several information security principles such as confidentiality, availability, and integrity. Feature selection methods have the potential to improve the classification of static analysis alerts and thereby reduce the false positive rates. Thus, the goal of this research effort was to improve the classification of static analysis alerts by proposing and testing a novel method leveraging feature selection. The proposed model was developed and subsequently tested on three open source PHP applications spanning several years. The results were compared to a classification model utilizing all features to gauge the classification improvement of the feature selection model. The model presented did result in the improved classification accuracy and reduction of the false positive rate on a reduced feature set. This work contributes a real-world static analysis dataset based upon three open source PHP applications. It also enhanced an existing data set generation framework to include additional predictive software features. However, the main contribution is a feature selection methodology that may be used to discover optimal feature sets that increase the classification accuracy of static analysis alerts.

Software development in the complex environment in the automotive industry puts high pressureon developers to develop high quality and robust software aligned to customers’ requirements. High quality software is foremost ensured by conducting software testing of the product under development. However, software testing in the automotive industry poses challenges of testing early in the development process, due to the limits of conducting tests in full-scaled vehicle environments. This challenge needs to be addressed for software development teams developing software that communicates with the complex on-board embedded system in vehicles. This study has been conducted in a case study approach at Scania CV AB in Södertälje in order to understand drivers to defects that emerge in finalized software products. Defects and drivers to defects found in finalized software products have been identified by conducting interviews with the SCPT-team responsible for the development of the product Escape. Escape is delivered to the production department and enables functions such as calibrating, set parameters, and run quality assurance tests on the on-board embedded system in vehicles. The identified defects and drivers have subsequently been discussed with experienced professionals and researchers within software testing. This provided applicable testing techniques and activities to undertake in order to address the identified drivers causing defects in finalized software products. The contribution of this study highlights the importance of incorporating software testing in early development phases in complex embedded systems as defects are more costly to correct the later they are identified. Static analysis tools have further been found to provide a suitable support to address the immense number of possible parameter combinations in vehicles. Furthermore, Software in the Loop environments have been found to be an applicable way of incorporating integration testing and system testing earlier in the development phase enabling identification of interoperability defects generally found late in the development process. Including persons responsible for testing the software in early requirements discussion has further been found to be of great importance as it minimizes the risk of misunderstandings between customers and developers. / Mjukvaruutveckling i den komplexa miljön bilindustrin befinner sig i sätter hög press på mjukvaruutvecklare att utveckla robusta mjukvaruprogram av hög kvalitet som uppfyller kundernas krav. Mjukvaruprogram av hög kvalitet är först och främst säkerhetsställd genom mjukvarutestning av produkten under utveckling. Däremot finns det en del utmaningar när det kommer till mjukvarutestning av mjukvaruprogram i bilindustrin på grund av den begränsade möjligheten till att testa programvaran i helbilsmiljöer. Team som utvecklar mjukvaruprogram som kommunicerar med det komplexa inbyggda systemet i fordon måste ta itu med denna utmaning. För att undersöka anledningar till att defekter identifieras i mjukvaruslutprodukter har denna studies tillvägagångssätt varit en fallstudie på Scania CV AB i Södertälje. Anledningar till defekter identifierade i slutprodukter har undersökts genom intervjuer med SPCT-teamet som ansvarar för att utveckla och testa produkten Escape. Escape är en produkt som används av produktionsavdelningen och erbjuder funktioner så som parametersättning, kalibrering och att köra kvalitetstester av det inbyggda systemet i fordon. De identifierade anledningarna till defekter har därefter diskuterats med erfarna mjukvarutestare inom både industrin och akademin. Det har bidragit till användbara testtekniker och testaktiviteter att ta sig an för att ta i tu med dem identifierade defekterna och dess anledningar som bidrar till defekter i slutprodukter. Forskningsbidraget från denna studie betonar hur viktigt det är att inkorporera mjukvarutestning tidigt i utvecklingsprocessen av komplexa inbyggda system eftersom defekter är dyrare att rätta till ju senare de upptäcks. Statiska analysverktyg har visat sig utgöra en användbar hjälp för att ta i tu med den stora mängden möjliga parameterkombinationer i fordon. Dessutom har Software in the Loop miljöer visat sig vara ett användbart sätt att möjliggöra integrationstestning och systemtestning tidigt i utvecklingsprocessen vilket kan identifiera defekter som vanligtvis först identifieras sent i utvecklingsprocessen. Involvera personer som är ansvariga för mjukvarutestning av produkten tidigt i kravdiskussioner har också visat sig vara viktigt för att minimera risken för missförstånd mellan kunder och utvecklare.

Software testing is becoming more prominent within the automotive industry due to more complex systems, and functions are implemented in the vehicles. The vehicles in the future will have the functionality to manage different levels of automation, which also means that vehicles driven by humans will have more supportive functionality to increase safety and avoid accidents. These functionalities result in a massive growth in the number of test scenarios to indicate that the vehicles are safe, and this makes it impossible to continue performing the tests in the same way as it has been done until today. The new conditions require that the test scenarios and Test Cases both be generated and executed automatically. In this thesis, an investigation and evaluation are performed to analyze the Automatic Test Case Generation methods available for inputs from Natural Language Requirements in an automotive industrial context at NEVS AB. This study aims to evaluate the NAT2TEST strategy by replacing the manual method and obtain a similar or better result. A comparative analysis is performed between the manual and automated approaches for various levels of requirements. The results show that utilizing this strategy in an industrial scenario can improve efficiency if the requirements to be tested are for well-documented lower-level requirements.

L'objectif de cette thèse est de contribuer à l'élaboration d'une méthode de test de systèmes mobiles. L'approche développée est fondée sur la description de tests à l'aide de scénarios et leurs vérifications sur une trace d'exécution. Un scénario modélise le comportement et les interactions que l'on souhaite observer entre un ensemble de nœuds. Les caractéristiques des systèmes mobiles nous ont conduit à représenter un scénario sous deux points de vue différents et complémentaires. Un premier représente des événements de communications entre les nœuds et un second représente la topologie des liens entre ces nœuds. Notre approche est décomposée en deux étapes : une étape de spécification des cas de tests à l'aide de scénarios et une étape de vérification de ces scénarios sur des traces d'exécutions. La première consiste à spécifier à l'aide du langage dédié TERMOS les cas de test de l'application mobile à vérifier. Ce langage TERMOS a été mis en œuvre au sein de l'atelier UML Papyrus. À partir des scénarios décrits de manière graphique, nous générons pour chacun d'eux un automate ainsi qu'une séquence de topologie que nous utilisons dans l'étape suivante. La deuxième étape consiste à vérifier chaque scénario sur des traces d'exécutions provenant de l'application à tester. Pour cela un premier outil recherche les occurrences de la séquence de topologie du scénario dans la trace d'exécution. Pour chacune d'entre elles, l'automate est exécuté et conclut à un verdict. L'analyse de l'ensemble des verdicts d'un scénario permet de détecter les fautes présentes dans le système. / The main objective of this thesis is to contribute to elaborating a mobile system test method. The proposed approach is based on test definition using scenarios and their verification on an execution trace. A scenario modelizes the behavior and the interactions we want to achieve on a set of nodes. Considering the characteristics of mobile systems we represented scenarios from two different but complementary points of view. The first represents communication events between nodes and the second represents the link topology between the nodes. Our approach is composed of two steps : a first step to specify the test cases by using scenarios and a second step to verify these scenarios on execution flows. The first step consists in using the dedicated TERMOS language in order to specify the test cases of the mobile application. The TERMOS language has been developed in the UML Papyrus workshop. Based on the graphically defined scenarios, we generate an automaton for each one of them, as well as a sequence of topologies which we will be using in the next step. The second step consists in verifying each scenario by using execution traces from the application to be tested. Therefore a first tool detects scenario topology sequences in the execution flow. For each one of them the automaton is executed and comes out with a verdict. The analysis of all the verdicts of a scenario allows the detection of faults in the system.

Software Quality is a key concern for any companies working with software development. This is true due to the fact that the success of any software directly depends on Quality of software. It is expected that the software is of best quality for a long duration of time. With the introduction of Mobile applications the task of maintaining the quality of an application has been difficult and have faced many challenges. Many companies working with mobile application have reformed their process in order to maintain the quality of their application. The introduction of Automation testing in the test process is one such reform that have changed the face of mobile application testing in today’s world. This work deals with the concepts of Automation System testing for the mobile application which is until now a new thing and it has many things yet to be explored. The approach to automation testing is simple yet unique for the department of PT-MT/Quality Management in Robert Bosch GmbH based in Leinfelden, Stuttgart. Over here a selection and implementation of a test framework will be done for Automation testing of the mobile Applications that are being developed. For this a requirement specification document is being created which will form the basis for selecting a framework from the KT Analysis table. Finally, a framework TestComplete will be implemented for the already developed application "PLR measure&go" The implementation will include all the procedure required to set up the test framework as a part of documentation. The framework TestComplete will be used to create System test for iOS and Android operation system. Lastly the execution of test and the Result reporting is being shown as a complete process for Automation testing.

Many software projects include an extensive suite of example-based unit tests. The examples in the test suite can be used as an implicit specification of the behavior of the software. Inferring invariants from these examples may aid in the creation of property-based tests. However, the existing research on this topic is scarce and there is none conducted using the inference of invariants. In this thesis, we examine software projects with existing test suites. The Daikon invariant detector is used to infer invariants from test executions. The resulting invariants are then used to formalize properties to be used in property-based tests. The success of the process depends on a few variables. First, the tests have to be descriptive enough to allow any generalization to take place. Second, the Daikon invariant detector needs to be able to create an abstraction from the examples that are not too restrictive nor too lenient. Third, the format of the invariants needs to match available value generator constraints in property-based testing frameworks. Fourth, the resulting tests need to provide some benefit to the test suite. In all experiments, at least one of these requirements is not met. The conclusion is that inferred invariants are likely not useable in the creation of property-based tests. Furthermore, in instances where properties can be derived from invariants, the resulting tests do not practically improve the test suite. For the test suite to be improved, the abstraction of examples needs to describe properties of the software not contained in the existing tests. / Många mjukvaruprojekt använder sig av automatiserade enhetstester. Dessa är ofta baserade på flertalet enskilda exempel. Dessa kan ses som en implicit specifikation. Detta skulle kunna förenkla processen att skapa egenskapsbaserade tester. Tidigare forskning inom detta område är sällsynt och inga tidigare studier använder invarianter för att skapa egenskapsbaserade tester. Invariantdetektionsprogrammet Daikon används för att dynamisk utläsning av invarianter från exekvering av tester. Dessa användas sedan för att formalisera programegenskaper till egenskapsbaserade tester. Denna tekniks framgång är beroende av flera variabler. Befintliga tester måste vara tillräckligt deskriptiva för att kunna abstraheras till generalla egenskaper. Daikon behöver kunna utläsa invarianter from testexekveringen som varken är för generella eller för specifika. Invarianternas format behöver likna generatorfunktioner i egenskapsbaserade testramverk. De skapade egenskapsbaserade testerna behöver bidra positivt till programmets testsvit. I alla experiment fanns brister i åtminstone ett av dessa steg. Således är slutsatsen att automatiskt utlästa invarianter sannolikt inte är ett effektivt sätt att skapa egenskapsbaserade tester. Dessutom, i de fall då egenskapsbaserade tester skapats bidrar de inte till en förbättring av testsviten. För att förbättra testsviten måste de nya testerna beskriva egenskaper som inte redan testas av de befintliga testerna.

A popular architecture for developing large-scale systems is the microservice architecture, which is currently in use by companies such as Amazon, LinkedIn, and Uber. The are many benefits of the microservice architecture with respect to maintainability, resilience, and scalability. However, despite these benefits, the microservice architecture presents its own unique set of challenges, particularly related to software testing. Software testing is exacerbated in the microservice architecture due to its complexity and distributed nature. To mitigate this problem, this project work investigated the use of a graph-based visualization system to simplify the software testing process of microservice systems. More specifically, the role of the visualization system was to provide an analysis platform for identifying the root cause of failing test cases. The developed visualization system was evaluated in a usability test with 22 participants. Each participant was asked to use the visualization system to solve five tasks. The average participant could on average solve 70.9% of the five tasks correctly with an average effort rating of 3.5, on a scale from one to ten. The perceived average satisfaction of the visualization system was 8.0, also on a scale from one to ten. The project work concludes that graph-based visualization systems can simplify the process of identifying the root cause of failing test cases for at least five different error types. The visualization system is an effective analysis tool that enables users to follow communication flows and pinpoint problematic areas. However, the results also show that the visualization system cannot automatically identify the root cause of failing test cases. Manual analysis and an adequate understanding of the microservice system are still necessary. / En populär arkitektur för att utveckla storskaliga system är mikrotjänstarkitekturen, som för närvarande används av företag som Amazon, LinkedIn och Uber. Det finns många fördelar med mikrotjänstarkitekturen med avseende på underhållbarhet, motståndskraft och skalbarhet. Men trots dessa fördelar presenterar mikrotjänstarkitekturen sin egen unika uppsättning utmaningar, särskilt med hänsyn till mjukvarutestningen. Mjukvarutestningen försvåras i mikrotjänstarkitekturen på grund av dess komplexitet och distribuerade natur. För att mildra detta problem undersökte detta projektarbete användningen av ett grafbaserat visualiseringssystem för att förenkla mjukvarutestprocessen för mikrotjänstsystem. Mer specifikt var visualiseringssystemets roll att tillhandahålla en analysplattform för att identifiera grundorsaken till misslyckade testfall. Det utvecklade visualiseringssystemet utvärderades i ett användbarhetstest med 22 deltagare. Varje deltagare ombads att använda visualiseringssystemet för att lösa fem uppgifter. Den genomsnittliga deltagaren kunde i genomsnitt lösa 70.9% av de fem uppgifterna korrekt med ett genomsnittligt ansträngningsbetyg på 3.5, på en skala från ett till tio. Den upplevda genomsnittliga nöjdheten med visualiseringssystemet var 8.0, också på en skala från ett till tio. Projektarbetet drar slutsatsen att grafbaserade visualiseringssystem kan förenkla processen att identifiera grundorsaken till misslyckade testfall för minst fem olika feltyper. Visualiseringssystemet är ett effektivt analysverktyg som gör det möjligt för användare att följa kommunikationsflöden och peka ut problemområden. Men resultaten visar också att visualiseringssystemet inte automatiskt kan identifiera grundorsaken till misslyckade testfall. Manuell analys och en grundlig förståelse av mikrotjänstsystemet är fortfarande nödvändigt.

The testing of software for robot controllers has become increasingly difficult as robotic systems become more complex. As the complexity of the systems increases, the number of hardware systems that the robot relies on also grows. This poses a challenge in testing robot controllers, which is crucial to ensure that robots function safely and effectively in their intended applications. While simulation can be used as a platform for software testing, it is not feasible to simulate everything in a virtual environment, especially when test cases require physical connections to hardware for input and output signals sent to robot controllers. Therefore, the objective of this thesis is to replicate I/O device network communication in order to enhance virtual testing processes. The approach employed involves capturing real-time network traffic, modifying and rebuilding it, and subsequently replaying it. The work examines existing academic research on these approaches and technologies, and investigates the specific challenges in the testing process by conducting research within a company leading globally in industrial robot development. A conceptual model is proposed, and a prototype is developed. The solution demonstrates potential in addressing the current challenges in robot controller testing by enabling network capture, modification, and level 4 network traffic replay. However, experimental results reveal various limitations, such as significant delays in generating responses. Therefore, further research and development are required if the solution is to be implemented in a real-world setting.