Context: Exploratory Testing (ET) and Scripted Testing (ST) are two of the more commonly practiced manual testing approaches in industry. ST is a traditional testing approach in which testing is carried out by executing pre-designed test cases. While in ET, learning, test designing and test execution are carried out simultaneously. In many instances, ET and ST complement each other very well in projects; however, proponents of ET claim that ET is more cost-beneficial in comparison to ST. Moreover, a few studies have indicated that ET is more effective in defect detection. Nevertheless, to the best of our knowledge, no study has been conducted to compare the costs and benefits of these two approaches. Objectives: The aim of this study was to conduct a qualitative Cost-Benefit Analysis (CBA) of ET in comparison with ST. By comparing and analyzing these two testing approaches, this study attempts to aid in decision-making with respect to how resources should be allocated for ET and ST for certain projects. Methods: The factors of costs and benefits of ET and ST were identified by conducting six semi-structured interviews in industry. Based on the analysis of these factors, a CBA model is proposed. The academic and industrial evaluation of the proposed CBA model was performed by conducting five interviews with researchers and practitioners. In addition, a qualitative CBA of a process of ET, Session-Based Testing Management (SBTM), and a process of ST, Test-Case Based Testing (TCBT) is conducted by collecting data through questionnaires and interviews with industry practitioners. A total of 22 questionnaire responses and seven interviews were analyzed. Results: By analyzing the identified cost and benefit factors, a CBA model was developed based on the testing phases stated in the ISO/IEC 29119 standard. A qualitative CBA of the SBTM process in comparison with the TCBT process was conducted by applying the CBA model in a questionnaire. The following findings were gathered from the CBA: • The differences of the SBTM and TCBT processes are identified by an analysis of the activities performed in various organizations, which the respondents belonged to. • The results of the analysis and comparison, of the costs (in terms of effort) and benefits (quality of the testing activities) of these two testing processes, are presented with respect to each testing phase. • The factors that impact the costs and benefits of using SBTM and TCBT, are summarized and discussed in this report. • The scenarios, in which SBTM and TCBT can be more cost-beneficial, are identified based on practitioners’ opinions. Conclusions: According to the survey results, industry practitioners consider SBTM as more cost-beneficial in comparison with TCBT, particularly in the test design, implementation and test execution phases. However, industry practitioners also stressed that ET should not be considered as a replacement for ST. In some contexts, testing objectives are better achieved through a more scripted approach, while, in other contexts, testing objectives will benefit more from the ability to create and improve tests as they are being executed. Whether a testing approach is valuable or cost-beneficial also depends on the context of project and the required benefits.

Software testing is considered as one of the key phases in the software-development life cycle (SDLC). The main objective of software testing is to detect the faults either through manual testing or with automated testing approach. The most commonly adopted software testing approach in industries is test case based testing (TCBT) which is usually done manually. TCBT is mainly used by the software testers to formalize and guide their testing activities and set theoretical principals for testing. On the other hand, model based testing (MBT) is widely used automation software testing technique to generate and execute the tests. Both techniques are showing their prominence in real time with some pros and cons. However, there is no formal comparison available between these two techniques. The main objective of this thesis work is to find out the difference in test cases in TCBT and MBT in terms of providing better test coverage ( Statement, Branch and Path), requirement traceability, cost and time. To fulfill the aims of the research we have conducted interviews for static validation, and later we did an experiment for validating those results dynamically. The analysis of experiment results showed that the requirement traceability in MBT generated test cases are very hard to make the test cases traceable to the requirements, particularly with the open-source tool Model J-Unit. However, this can be done by using other commercial tools like Microsoft Spec Explorer or Conformiq Qtronic. Furthermore, we found by conducting experiment, that MBT consumes less time thus it is cost-effective as compared to TCBT and also MBT show better test coverage than TCBT. Moreover, we found that, in our case, requirement traceability is better in traditional TCBT approach as compared to MBT. / +4746851975