Software testing is a vital part of the software development process and is used for the purposes of quality assurance, reliability estimation and verification and validation. However, software testing is extremely costly and time consuming. For instance, prior research indicated that more than 50% of the software development cost is devoted to testing. Due to the increased use of the Unified Modeling Language (UML) and model driven architecture approach in systems analysis and design, Model-Based Testing has been discussed as a prominent solution for software testing to address the above problems. This study presents a methodology for determining the test path which can then be further used to determine the test case for Model-Based Testing. Three real-world cases are used to test the usability (including the concepts, application, and advantages) of the proposed methodology. With this approach, software errors can be found at the systems analysis and design stage and thereby reduce the cost of software testing and enhance the efficiency of system development.

Testing is the dominant technique for quality assurance of software systems. It typically consumes considerable resources in development projects, and is often performed in an ad hoc manner. This thesis is concerned with model-based testing, which is an approach to make testing more systematic and more automated. The general idea in model-based testing is to start from a formal model, which captures the intended behavior of the software system to be tested. On the basis of this model, test cases can be generated in a systematic way. Since the model is formal, the generation of test suites can be automated and with adequate tool support one can automatically quantify to which degree they exercise the tested software. Despite the significant improvements on model-based testing in the last 20 years, acceptance by industry has so far been limited. A number of commercially available tools exist, but still most testing in industry relies on manually constructed test cases. This thesis address this problem by presenting a methodology and associated tool support, which is intended to be used for model-based testing of communication protocol implementations in industry. A major goal was to make the developed tool suitable for industrial usage, implying that we had to consider several problems that typically are not addressed by the literature on model-based testing. The thesis presents several technical contributions to the area of model-based testing, including - a new specification language based on the functional programming language Erlang, - a novel technique for specifying coverage criteria for test suite generation, and - a technique for automatically generating test suites. Based on these developments, we have implemented a complete tool chain that generates and executes complete test suites, given a model in our specification language. The thesis also presents a substantial industrial case study, where our technical contributions and the implemented tool chain are evaluated. Findings from the case study include that test suites generated using (model) coverage criteria have at least as good fault-detection capability as equally large random test suites, and that model-based testing could discover faults in previously well-tested software where previous testing had employed a relaxed validation of requirements.

A case study for the tool LBTest illustrating benets and limitations of the tool along the terms of usability, results and costs. The study shows the use of learning based testing on a safety critical application in the avionics industry. While requiring the user to have the oretical knowledge of the tools inner workings, the process of using the tool has benefits in terms of requirement analysis and the possibility of finding design and implementation errors in both the early and late stages of development

Software testing is a vital part of the software development process, usually implemented at the coding stage, and costly. Due to the increased use of the Unified Modeling Language (UML) and Model Driven Architecture (MDA) approach in systems analysis and design, Model-Based Testing has been discussed as a prominent solution for software testing to reduce the cost of software testing. Prior researches proposed an integrated method which utilizes the artifacts from the Platform Independent Model (PIM) to construct the test paths and generate the test cases. This study develops a methodology which extracting the information from PIM (e.g., Sequence Diagram and Class diagram) to generate the test cases directly. The research methodology is articulated using the design science research methodology. A usability evaluation is performed to demonstrate its applicability. With this methodology, the test cases can be easily generated; thereby reducing the cost and enhancing the efficiency of Model-Based Testing.

Software testing is a vital part of the software development process and is costly. Due to the increased use of the unified modeling language and model driven architecture approach in systems analysis and design, model-based testing has been discussed as a prominent solution for software testing to reduce the cost of software testing. Prior researches proposed an integrated method which utilizes the artifacts from the Platform Independent Model (PIM) to construct the test path and constrained class tuples. These two can then be integrated to construct an Integrated Testing Model (ITM) for determining the test data and test cases for Model-Based Testing. This study develops a methodology which extracting the information from ITM to construct the test cases. The research methodology is articulated using the design science research methodology. A prototype embedded methodology has been developed to automatically generate the test cases. A usability evaluation is performed on the prototype to demonstrate its usability. With this methodology, the test cases can be generated automatically; thereby reducing the cost and enhancing the efficiency of Model-Based Testing.

Currently, the increasing complexity of software and the short release cycles are becoming a challenge for testing software in an efficient and effective way. Traditionally, creating tests is done manually by engineers, which are then automatically or manually executed on the actual software. Manually creating test cases is a time-consuming effort. For the last couple of decades, researchers have proposed ways to improve this process by automating parts of the testing steps. One of these approaches that have gained a lot of attention is called Model-Based Testing (MBT). MBT has been suggested as a way of automatically creating tests at a lower cost. Nonetheless, it is not very well studied how MBT is actually applied in industrial contexts and how these tests compare to manually written ones. This is particularly true for industrial control software such as the one found in the train domain, where strict requirements on testing are in place. In this thesis, we investigate the literature and review the related work on case studies on the MBT use in industry and its evaluation. We perform a case study to evaluate MBT on a train control management system provided by Bombardier Transportation. We use Comformiq Creator MBT Tool to create models for functional requirements of a master controller function and generate test cases. We provide the result of the modeling approach as well as a comparison between automatic test cases created by Conformiq Creator and manual test cases written by industrial engineers at Bombardier Transportation using the following metrics: test coverage and time spent on testing. The results of this comparison suggest that test coverage of MBT is higher and test cases are more detailed than manual testing. Our results are not conclusive in regard to the cost of using MBT, mainly because this depends on different testing scenarios and how testing is performed. We show that MBT is a suitable approach for modeling the functional requirements of a realistic industrial control software function. In this thesis work, we focus on system-level testing. As future work, applying MBT on lower levels of testing can be a promising way forward for evaluation. In addition, the transformation of these test cases into executable test scripts and the possible problems needs to be investigated further.

Background Requirements engineering and verification (REV) processes play es-sential roles in software product development. There are physical and non-physicaldistances between entities (actors, artifacts, and activities) in these processes. Cur-rent practices that reduce the distances, such as automated testing and alignmentof document structure and tracing only partially close the above mentioned gap.Objective The aim of this thesis is to investigate solutions w.r.t their abilityto reduce the distances between requirements engineering and verification. Twotechniques that are explored in this thesis are automated testing (model-basedtesting, MBT) and alignment of document structure and tracing (traceability).Method The research methods used in this thesis are systematic mapping, soft-ware requirements mining, case study, literature survey, validation study, and de-sign science.Results MBT and traceability are effective in reducing the distance between re-quirements and verification. However, both activities have some shortcoming thatneeds to be addressed when used for that purpose. Current MBT techniques inthe context of software performance do not attain all the goals of MBT: 1) require-ments validation, 2) checking the testability of requirements, and 3) the generationof an efficient test suite. These goals are essential to reduce the distance. We de-veloped and assessed performance requirements verification and test environmentgeneration approach to tackle these shortcomings. Also, traceability between re-quirements and verification suffers from the low granularity of trace links and doesnot support the verification of all requirements. We propose the use of taxonomictrace links to trace and align the structure of requirements specifications and ver-ification artifacts. The results from the validation study show that the solution isfeasible in practice. However, this comes with challenges that need to be addressed.Conclusion MBT and improved traceability reduce multiple distances betweenactors, artifacts, and activities in the requirements engineering and verificationprocess. MBT is most effective in reducing the distances when the model used isbuilt from the requirements. Traceability is essential in easing access to relevantinformation when needed and should not be seen as an overhead. When creatingtrace links, we need to consider the difference in the abstraction, structure, andtime between the linked artifacts / <p>Chapter 3 and 4 are papers submitted to journals, and therefore removed from the fulltext file.</p>

Yes / This paper concerns the systematic testing of robotic control software based on state-based models. We focus on cyclic systems that typically receive inputs (values from sensors), perform computations, produce outputs (sent to actuators) and possibly change state. We provide a testing theory for such cyclic systems where time can be represented and probabilities are used to quantify non-deterministic choices, making it possible to model probabilistic algorithms. In addition, refusals, the inability of a system to perform a set of actions, are taken into account. We consider several possible testing scenarios. For example, a tester might only be able to passively observe a sequence of events and so cannot check probabilities, while in another scenario a tester might be able to repeatedly apply a test case and so estimate the probabilities of sequences of events. These different testing scenarios lead to a range of implementation relations (notions of correctness). As a consequence, this paper provides formal definitions of implementation relations that can form the basis of sound automated testing in a range of testing scenarios. We also validate the implementation relations by showing how observers can be used to provide an alternative but equivalent characterisation. / This work has been supported by EPSRC, United Kingdom grant EP/R025134/2 RoboTest: Systematic Model-Based Testing and Simulation of Mobile Autonomous Robots, the Spanish MINECO-FEDER grant PID2021- 122215NB-C31 (AwESOMe) and the Region of Madrid grant S2018/TCS-4314 (FORTE-CM) co-funded by EIE Funds of the European Union.

<p>Model Based Testing (MBT) is a new-age test automation technique traditionally used for Functional Black-Box Testing. Its capability of generating test cases by using model developed from the analysis of the abstract behavior of the System under Test is gaining popularity. Many commercial and open source MBT tools are available currently in market. But each one has its own specific way of modeling and test case generation mechanism that is suitable for varied types of systems. Ericsson, a telecommunication equipment provider company, is currently adapting Model Based Testing in some of its divisions for functional testing. Those divisions haven’t yet attempted adapting Model Based Testing for non-functional testing in a full-pledged manner. A comparative study between various MBT tools will help one of the Ericsson’s testing divisions to select the best tool for adapting to its existing test environment. This also helps in improving the quality of testing while reducing cost, time and effort. This thesis work helps Ericsson testing division to select such an effective MBT tool. Based on aspects such as functionality, flexibility, adaptability, performance etc., a comparative study is carried out on various available MBT tools and a few were selected among them: Qtronic, ModelJUnit and Elvior Motes.This thesis also helps to understand the usability of the selected tools for modeling of non-functional requirements using a new method. A brief idea of modeling the non-functional requirements is suggested in this thesis. A System under Test was identified and its functional behavior was modeled along with the non functional requirements in Qtronic and ModelJUnit. An experimental analysis, backed by observations of using the new proposed method indicates that the method is efficient enough to carry out modeling non-functional requirements along with modeling of functional requirements by identifying the appropriate approach.Model Based Testing (MBT) is a new-age test automation technique traditionally used for Functional Black-Box Testing. Its capability of generating test cases by using model developed from the analysis of the abstract behavior of the System under Test is gaining popularity. Many commercial and open source MBT tools are available currently in market. But each one has its own specific way of modeling and test case generation mechanism that is suitable for varied types of systems. Ericsson, a telecommunication equipment provider company, is currently adapting Model Based Testing in some of its divisions for functional testing. Those divisions haven’t yet attempted adapting Model Based Testing for non-functional testing in a full-pledged manner. A comparative study between various MBT tools will help one of the Ericsson’s testing divisions to select the best tool for adapting to its existing test environment. This also helps in improving the quality of testing while reducing cost, time and effort. This thesis work helps Ericsson testing division to select such an effective MBT tool. Based on aspects such as functionality, flexibility, adaptability, performance etc., a comparative study is carried out on various available MBT tools and a few were selected among them: Qtronic, ModelJUnit and Elvior Motes.</p><p>This thesis also helps to understand the usability of the selected tools for modeling of non-functional requirements using a new method. A brief idea of modeling the non-functional requirements is suggested in this thesis. A System under Test was identified and its functional behavior was modeled along with the non functional requirements in Qtronic and ModelJUnit. An experimental analysis, backed by observations of using the new proposed method indicates that the method is efficient enough to carry out modeling non-functional requirements along with modeling of functional requirements by identifying the appropriate approach.</p>

<p>Testing is a critical activity in the software development process in order to obtain systems of high quality. Tieto typically develops complex systems, which are currently tested through a large number of manually designed test cases. Recent development within software testing has resulted in methods and tools that can automate the test case design, the generation of test code and the test result evaluation based on a model of the system under test. This testing approach is called model-based testing (MBT).</p><p>This thesis is a feasibility study of the model-based testing concept and has been performed at the Tieto office in Karlstad. The feasibility study included the use and evaluation of the model-based testing tool Qtronic, developed by Conformiq, which automatically designs test cases given a model of the system under test as input. The experiments for the feasibility study were based on the incremental development of a test object, which was the client protocol module of a simplified model for an ATM (Automated Teller Machine) client-server system. The experiments were evaluated both individually and by comparison with the previous experiment since they were based on incremental development. For each experiment the different tasks in the process of testing using Qtronic were analyzed to document the experience gained as well as to identify strengths and weaknesses.</p><p>The project has shown the promise inherent in using a model-based testing approach. The application of model-based testing and the project results indicate that the approach should be further evaluated since experience will be crucial if the approach is to be adopted within Tieto’s organization.</p>