Testing is a part of the software development life cycle (SDLC) which ensures the quality and efficiency of the software. It gives confident to the developers about the system by early detecting faults of the system. Therefore, it is considered as one of the most important part of the SDLC. Unfortunately, testing is often neglected by the developers mainly because of the time and cost of the testing process. Testing involves lots of manpower, specially for a large system, such as distributed system. On the other hand, it is more common to have bugs in a large system than a small centralized system and therefore there is no alternative of testing to find and fix the bugs. The situation gets worst if the developer follows one of the most powerful development process called continuous integration process. This is because developers need to write the test cases in each cycle of the continuous integration process which increase the development time drastically. As a result, testing often neglected for large systems. This is an alarming situation because distributed system is one of the most popular and widely accepted system in both industries and academia. Therefore, this is one of the highly pressured areas where lot of developers engaged to provide distributed software solutions. If these systems delivered to the users untested, there is a high possibility that we will end up with a lot of buggy systems every year. There are also a very few number of testing framework exist in the market for testing distributed system compared to the number of testing framework exists for traditional system. The main reason behind this is, testing a distributed system is far difficult and complex process compares to test a centralized system. Most common technique to test a centralized system is to test the middleware which might not be the case for distributed system. Unlike the traditional system, distributed system can be resides in multiple location of different corners of the world. Therefore, testing and verification of distributed systems are difficult. In addition to this, distributed systems have some basic properties such as fault tolerance, availability, concurrency, responsiveness, security, etc. which makes the testing process more complex and difficult. This research proposed a multi-agent based testing framework for distributed system where multiple agent communicate with each other and accomplish the whole testing process for a distributed system. The bullet proof idea of testing centralizes system has been reused partially to design the framework so that developers will be more comfortable to use the framework. The research also focused on the automation of testing process which will reduce the time and cost of the whole testing process and relief the developer from re-generating the same test cases over and over before each release of the application. This paper briefly described the architecture of the framework and communication process between multiple agents.

In the field of industrial automation there is a huge demand for real time communication networks and there are several different protocols like EtherCAT, PROFINET IO, SERCOS competing each other in the market. Many of the products in this industry are subjected to hard real-time communication requirements. The purpose of this thesis is in introducing tools to automatically test the various requirements that are helpful in deciding the performance of real time communication systems. Developing a well-defined test framework is one of the important tasks in this thesis project. The end users of the test framework can focus more on managing and analyzing the results from the framework instead of the design process. The thesis work presents the test requirements, design of test system and automating the measurement process by selecting appropriate hardware. The report also explains the design of commands, method used for communication between different systems and also discusses the different methods that could be applied for measuring the performance, the limitations of some of these methods when applied to the framework. The architecture and the working of the framework is covered in later chapters. The framework uses EtherCAT master communication stack developed in-house at ABB. EtherCAT is just used as a pilot test case, but in general the framework could be applied to other Ethernet based industrial communication protocols with suitable hardware or software modifications.

Penetration testing is widely used in industry as a test method for web application security assessment. However, penetration testing is often performed late in a software development life cycle as an isolated task and usually requires specialized security experts. There is no well-defined test framework providing guidance and support to general testers who usually do not have in-depth security expertise to perform a systematic and cost-efficient penetration test campaign throughout a security-oriented software development life cycle. In this thesis, we propose a model-driven penetration test framework for web applications that consists of a penetration test methodology, a grey-box test architecture, a web security knowledge base, a test campaign model, and a knowledge-based PenTest workbench. The test framework enables general testers to perform a penetration test campaign in a model-driven approach that is fully integrated into a security-oriented software development life cycle. Security experts are still required to build up and maintain a web security knowledgebase for test campaigns, but the general testers are capable of developing and executing penetration test campaigns with reduced complexity and increased reusability in a systematic and cost-efficient approach. A prototype of the framework has been implemented and applied to three web applications: the benchmark WebGoat web application, a hospital adverse event management system (AEMS), and a palliative pain and symptom management system (PAL-IS). An evaluation of the test framework prototype based on the case studies indicates the potential of the proposed test framework to improve how penetration test campaigns are performed and integrated into a security-oriented software development life cycle.

Penetration testing is widely used in industry as a test method for web application security assessment. However, penetration testing is often performed late in a software development life cycle as an isolated task and usually requires specialized security experts. There is no well-defined test framework providing guidance and support to general testers who usually do not have in-depth security expertise to perform a systematic and cost-efficient penetration test campaign throughout a security-oriented software development life cycle. In this thesis, we propose a model-driven penetration test framework for web applications that consists of a penetration test methodology, a grey-box test architecture, a web security knowledge base, a test campaign model, and a knowledge-based PenTest workbench. The test framework enables general testers to perform a penetration test campaign in a model-driven approach that is fully integrated into a security-oriented software development life cycle. Security experts are still required to build up and maintain a web security knowledgebase for test campaigns, but the general testers are capable of developing and executing penetration test campaigns with reduced complexity and increased reusability in a systematic and cost-efficient approach. A prototype of the framework has been implemented and applied to three web applications: the benchmark WebGoat web application, a hospital adverse event management system (AEMS), and a palliative pain and symptom management system (PAL-IS). An evaluation of the test framework prototype based on the case studies indicates the potential of the proposed test framework to improve how penetration test campaigns are performed and integrated into a security-oriented software development life cycle.

Penetration testing is widely used in industry as a test method for web application security assessment. However, penetration testing is often performed late in a software development life cycle as an isolated task and usually requires specialized security experts. There is no well-defined test framework providing guidance and support to general testers who usually do not have in-depth security expertise to perform a systematic and cost-efficient penetration test campaign throughout a security-oriented software development life cycle. In this thesis, we propose a model-driven penetration test framework for web applications that consists of a penetration test methodology, a grey-box test architecture, a web security knowledge base, a test campaign model, and a knowledge-based PenTest workbench. The test framework enables general testers to perform a penetration test campaign in a model-driven approach that is fully integrated into a security-oriented software development life cycle. Security experts are still required to build up and maintain a web security knowledgebase for test campaigns, but the general testers are capable of developing and executing penetration test campaigns with reduced complexity and increased reusability in a systematic and cost-efficient approach. A prototype of the framework has been implemented and applied to three web applications: the benchmark WebGoat web application, a hospital adverse event management system (AEMS), and a palliative pain and symptom management system (PAL-IS). An evaluation of the test framework prototype based on the case studies indicates the potential of the proposed test framework to improve how penetration test campaigns are performed and integrated into a security-oriented software development life cycle.

Penetration testing is widely used in industry as a test method for web application security assessment. However, penetration testing is often performed late in a software development life cycle as an isolated task and usually requires specialized security experts. There is no well-defined test framework providing guidance and support to general testers who usually do not have in-depth security expertise to perform a systematic and cost-efficient penetration test campaign throughout a security-oriented software development life cycle. In this thesis, we propose a model-driven penetration test framework for web applications that consists of a penetration test methodology, a grey-box test architecture, a web security knowledge base, a test campaign model, and a knowledge-based PenTest workbench. The test framework enables general testers to perform a penetration test campaign in a model-driven approach that is fully integrated into a security-oriented software development life cycle. Security experts are still required to build up and maintain a web security knowledgebase for test campaigns, but the general testers are capable of developing and executing penetration test campaigns with reduced complexity and increased reusability in a systematic and cost-efficient approach. A prototype of the framework has been implemented and applied to three web applications: the benchmark WebGoat web application, a hospital adverse event management system (AEMS), and a palliative pain and symptom management system (PAL-IS). An evaluation of the test framework prototype based on the case studies indicates the potential of the proposed test framework to improve how penetration test campaigns are performed and integrated into a security-oriented software development life cycle.

This bachelor thesis presents a comparison of three Java testing frameworks - JUnit 5,TestNG and Spock - with the purpose of evaluating their suitability in testing RESTAPIs built with Spring Boot. As the demand for reliable and high-quality softwaresystems continues to grow, automated testing techniques are crucial in ensuring thecorrect functionality of applications. Our study aims to fill the knowledge gap in thecurrent literature by focusing on unit tests for Java REST APIs running on the Springframework.We developed a single Spring Boot application and applied tests written using thethree selected testing frameworks. We then compared the performance of the frame-works based on execution time, memory usage and code conciseness. Additionally,we conducted a questionnaire to gather developer preferences for the frameworks.Our findings reveal that TestNG outperforms JUnit 5 in terms of performance, whileSpock requires fewer characters, making it more concise. However, JUnit 5 remainsthe most well-known and widely used testing framework among developers. Theresults of our study provide valuable insights into the performance and developerpreferences of the selected testing frameworks.

No description available.

As a consequence of the highly increasing cross-linking of computer systems in computer networks, the possibilities for accessing programs operated at these machines is becoming more and more independent from the possibilities of having physical access to them. Thus the former existing physical access controls have to be replaced by logical access controls which ensure that computer systems are only used for the intended purpose and that the stored data are handled securely and confidentially. The efficiency of such logical protection mechanism is verified by applying software security tests. During such tests it is proved whether security functions can be bypassed especially by exploiting software errors. In this diploma thesis approaches for the automation of software security tests are examined regarding their effectiveness and applicability. The results are used to introduce a requirement and evaluation model for the qualitative analysis of such security evaluation automation approaches. Additionally, the assertion is made that a highly automated software security evaluation is not a sensible development goal referring to the estimated cost-benefit ratio which is gained by trying to realise this goal. Based on this assertion, this diploma thesis discusses how to join the capabilities of a human tester and a software evaluation assistance system in an efficient test process. Based on this considerations, the design and implementation of a software security evaluation system which has been developed prototypically for this diploma thesis is described. This system significantly involves the human tester in the evaluation process but provides approaches for automation where possible. Furthermore this proof-of-concept prototype is evaluated regarding its practical applicability. / Durch die zunehmende starke Vernetzung von Computertechnologie wird die Möglichkeit des Zugriffs auf einzelne Computersysteme und den darauf ablaufenden Programmen zunehmend ebenso stark unabhängig von den physischen Zugangsmöglichkeiten des Zugreifenden zu diesen Systemen. Diese wegfallenden physischen Zugangsbarrieren müssen deshalb durch logische Zugriffsbeschränkungen ersetzt werden, die sicherstellen, dass Computersysteme nur zu den vorgesehen Zwecken verwendet und die darin gespeicherten Daten sicher und vertraulich verarbeitet werden. Die Wirksamkeit dieser logischen Schutzmechanismen wird mit Hilfe von s.g. Softwaresicherheitstests verifiziert. Dabei wird insbesondere überprüft, inwiefern Schutzfunktionen durch Zuhilfenahme von in der Software vorhandenen Programmfehlern umgangen werden können. Diese Diplomarbeit überprüft bestehende Ansätze für die Automatisierung solcher Sicherheitstests hinsichtlich ihrer Wirksamkeit und Anwendbarkeit. Aus den Resultaten dieser Untersuchung wird ein allgemeines Anforderungs- und Bewertungsmodell entwickelt, welches die qualitative Bewertung von Ansätzen zur Sicherheitstestautomatisierung zulässt. Desweiteren wird die Behauptung aufgestellt, dass die Forderung nach einer zu starken Automatisierung des Testverfahrens sich ungünstig gegenüber des Kosten-Nutzen-Verhältnisses auswirkt, welches bei der Realisierung dieser Forderungen zu erwarten ist. Darauf aufbauend versucht die Diplomarbeit abzugrenzen, wie sich die Fähigkeiten des menschlichen Testers und eines teilautomatisierbaren Softwaresystems effizient in einem Sicherheitstestprozess verbinden lassen. Basierend auf diesen Betrachtungen wird beschrieben, wie ein Sicherheitsevaluierungssystem, welches prototypisch für diese Diplomarbeit entwickelt wurde, den Menschen zur Erstellung von Testalgorithmen einbindet aber, wenn dies zweckmäßig ist, Automatisierungen ermöglicht. Dieses System wird daraufhin auf seine praktische Anwendbarkeit untersucht.

Model Driven Development (MDD) and Model Based Testing (MBT) are gaining inroads individually for their application in embedded software engineering projects. However, their full-edged and integrated usage in real-life embedded software engineering projects (e.g. industrially relevant examples) and executing MBT in resource constrained embedded systems (e.g. 16 bit system/64 KiByte memory) are emerging fields. Addressing the aforementioned gaps, this thesis proposes an integrated model-based approach and test framework for executing the model-based test cases, with minimal overhead, in embedded systems. Given a chosen System Under Test (SUT) and the system design model, a test framework generation algorithm generates the necessary artifacts (i.e., the test framework) for executing the model-based test cases. The main goal of the test framework is to enable test automation and test case execution at the host computer (which executes the test harness), thereby only the test input data is executed at the target. Significant overhead involved in interpreting the test data at the target is eliminated, as the test framework makes use of a target debugger (communication and decoding agent) on the host and a target monitor (software-based runtime monitoring routine) in the embedded system. In the prototype implementation of the proposed approach, corresponding (standardized) languages such as the Unified Modeling Language (UML) and the UML Testing Profile (UTP) are used for the MDD and MBT phases respectively. The applicability of the proposed approach is demonstrated using an experimental evaluation (of the prototype) in real-life examples. The empirical results indicate that the total time spent for executing the test cases in the target (runtime-time complexity), comprises of only the time spent to decode the test input data by the target monitor and execute it in the embedded system. Similarly, the only memory requirement in the target for executing the model-based test cases in the target is that of the software-based target monitor. A quantitative comparison on the percentage change in the memory overhead (runtime-memory complexity) for the existing approach and the proposed approach indicates that the existing approach (e.g. in a MDD/MBT tool-Rhapsody), introduces approximately 150% to 350% increase in memory overhead for executing the test cases. On the other hand, in the proposed approach, the target monitor is independent of the number of test cases to be executed and their complexity. Hence, the percentage change in the memory overhead for the proposed approach shows a declining trend w.r.t the increasing code-size for equivalent application scenarios (approximately 17% to 2%). Thus, the proposed test automation approach provides the essential benefit of executing model- based tests, without downloading the test harness in the target. It is demonstrated that it is feasible to execute the test cases specified at higher abstraction levels (e.g. using UML sequence diagrams) in resource constrained embedded systems and how this may be realized using the proposed approach. Further, as the proposed runtime monitoring mechanism is time and memory-aware, the overhead parameters can be accommodated in the earlier phases of the embedded software development cycle (if necessary) and the target monitor can be included in the final production code. The aforementioned advantages highlight the scalability, applicability, reliability and superiority of the proposed approach over the existing methodologies for executing the model-based test cases in embedded systems.