<p>Testing is the dominant verification technique used in the software industry today. The use of automatic test case execution increases, but the creation of test cases remains manual and thus error prone and expensive. To automate generation and selection of test cases, model-based testing techniques have been suggested.</p><p>In this thesis two central problems in model-based testing are addressed: the problem of how to formally specify coverage criteria, and the problem of how to generate a test suite from a formal timed system model, such that the test suite satisfies a given coverage criterion. We use model checking techniques to explore the state-space of a model until a set of traces is found that together satisfy the coverage criterion. A key observation is that a coverage criterion can be viewed as consisting of a set of items, which we call coverage items. Each coverage item can be treated as a separate reachability problem. </p><p>Based on our view of coverage items we define a language, in the form of parameterized observer automata, to formally describe coverage criteria. We show that the language is expressive enough to describe a variety of common coverage criteria described in the literature. Two algorithms for test case generation with observer automata are presented. The first algorithm returns a trace that satisfies all coverage items with a minimum cost. We use this algorithm to generate a test suite with minimal execution time. The second algorithm explores only states that may increase the already found set of coverage items. This algorithm works well together with observer automata.</p><p>The developed techniques have been implemented in the tool CoVer. The tool has been used in a case study together with Ericsson where a WAP gateway has been tested. The case study shows that the techniques have industrial strength.</p>

Naujuose programiniuose produktuose plačiai taikoma mobilioji įranga (mobilieji telefonai, delniniai kompiuteriai). Programinės įrangos kūrimo procese svarbią vietą užima testavimas. Dabartinių mobiliųjų įranginių apribojimai (darbo sparta, atminties kiekis, energija, ekrano dydis, platformų įvairumas) kelia naujas problemas programinės įrangos kūrimo procesui, tame tarpe ir testavimui. Testavimo proceso automatizavimas leidžia sumažinti bandymų trukmę, padidinti testavimo darbų apimtį. Programinės įrangos testavime mobiliems įrenginiams svarbu išskirti grafinės vartotojo sąsajos testavimą. Svarbi testavimo dalis – testavimas pagal specifikaciją. Šiame darbe nagrinėjamas grafinės vartotojo sąsajos automatizuotas testavimas mobiliai įrangai, nustatomi tikslai, galimi keliai jiems pasiekti bei galimas testavimo aplinkos praplėtimas testų generavimui pagal specifikaciją. / Mobile devices such as cell phones and personal digital assistance are widely used in new software products. Testing takes important place in software development process. Constraints of mobile devices (speed, amount of memory, energy, small screen, wide range of platforms) raise new problems for software development process including testing phase. Automated approach of software testing reduces testing time and increases testing range. It is important to distinguish graphic user interface as a special part of testing. The important part of testing is specification based testing. The aim of these master theses is to analyze automated testing of GUI for mobile devices, define testing tasks and enhance mobile device testing framework by providing means for test case generation from specification.

Made available in DSpace on 2016-08-17T14:53:23Z (GMT). No. of bitstreams: 1 Dissertacao Jessica Bassani.pdf: 6480289 bytes, checksum: 763e127476700fc704634c924dde4d34 (MD5) Previous issue date: 2012-12-21 / Conselho Nacional de Desenvolvimento Científico e Tecnológico / We present an approach to support the creation of test cases for software systems in cloud computing environments. This approach is based on Model Driven Engineering (MDE). A framework, a methodology and metamodels are proposed to support the generation of test cases. Metamodels for cloud computing environments and transformation definitions are proposed. Two illustrative examples help to understand our approach and the steps to apply it. A prototype implementing the proposed framework is presented and it works together with MT4MDE and SAMT4MDE to generate semi-automatically transformation definitions and contribute to the semiautomatic creation of testing model and test code for cloud computing platforms. We present an evaluation of our approach and compare it with other approaches, detaching main differences, including advantages and disavantages. / Apresenta-se uma abordagem para suportar a criação de casos de teste para sistemas de software em ambientes de Computação em Nuvem. Esta abordagem é baseada na Engenharia Dirigida por Modelos (MDE). Um framework, uma metodologia e metamodelos são propostos para suportar a geração dos casos de teste. Metamodelos específicos para plataformas de Computação em Nuvem são propostos. Dois exemplos ilustrativos ajudam a compreender a abordagem e os passos para aplicá-la. A prototipagem do framework proposto reutiliza as ferramentas MT4MDE e SAMT4MDE para gerar definições de transformação semiautomaticamente para contribuir com a criação semiautomática de modelos de teste e códigos de teste para plataformas de Computação em Nuvem. Uma avaliação é feita, comparando com o framework desenvolvido abordagens existentes com nossa abordagem proposta, destacando os principais diferenciais, incluindo vantagens e desvantagens.

To produce high quality software both software developers and testers need continuous improvement in their work methodologies and processes. So, far much work has been done in the effective ways of eliciting and documenting the requirements. However important aspect is to make sure that whatever is documented in specifications actually works correctly in the developed software. Software testing is done to ensure this phenomenon. Aim of this thesis is to develop a software test case work flow strategy that helps in identification and selection of suitable test paths that can be used as an input to acceptance testing and as a pre-requisite to start actual testing of the system. This thesis focuses on organizing system test artifacts by closely specifying them with system requirements and use cases. In this perspective focus of this thesis is on requirement writing by use cases, requirements traceability, test case prioritization and application acceptance criteria. A structured way to design test cases is proposed with the help of use cases. Some work is done to trace user needs to system requirements and use cases and benefits of using use case modeling approach in structuring the relationships among test cases is analyzed. As test cases are subject to changes in future so, challenges imposed due to traceability among requirements, use cases and test cases are main subjects of this work along with the challenges faced by software testers to perform application acceptance testing. A green path scheme is proposed to help testers define application acceptance criteria and weight assignment approach is used to prioritize the test cases and to determine the percentage of application running successfully.

Verification and validation is one of the most important concerns in the area of software engineering towards more reliable software development. Hence it is important to overcome the challenges of testing concurrent programs. The extensive use of concurrent systems warrants more attention to the concurrent software testing. For testing concurrent software, automatic tools development is getting increased focus. The first part of this study presents a systematic review that aims to explore the state-of-the-art of concurrent software testing. The systematic review reports several issues like concurrent software characteristics, bugs, testing techniques and tools, test case generation techniques and tools, and benchmarks developed for the tools. The second part presents the evaluation of four commercial and open source static analysis tools detecting Java multithreaded bugs. An empirical evaluation of the tools would help the industry as well as the academia to learn more about the effectiveness of the static analysis tools for concurrency bugs.

Context: Software testing is expensive, labor intensive and consumes lot of time in a software development life cycle. There was always a need in software testing to decrease the testing time. This also resulted to focus on Automated Software Testing (AST), because using automated testing, with specific tools, this effort can be dramatically reduced and the costs related with testing can decrease [11]. Manual Testing (MT) requires lot of effort and hard work, if we measure in terms of person per month [11]. Automated Software testing helps to decrease the work load by giving some testing tasks to the computers. Computer systems are cheap, they are faster and don‘t get bored and can work continuously in the weekends. Due to this advantage many researches are working towards the Automation of software testing, which can help to complete the task in less testing time [10]. Objectives: The main aims of this thesis is to 1.) To systematically classify contributions within AST. 2.) To identify the different benefits and challenges of AST. 3.) To identify the whether the reported benefits and challenges found in the literature are prevalent in industry. Methods: To fulfill our aims and objectives, we used Systematic mapping research methodology to systematically classify contributions within AST. We also used SLR to identify the different benefits and challenges of AST. Finally, we performed web based survey to validate the finding of SLR. Results: After performing Systematic mapping, the main aspects within AST include purpose of automation, levels of testing, Technology used, different types of research types used and frequency of AST studies over the time. From Systematic literature review, we found the benefits and challenges of AST. The benefits of AST include higher product quality, less testing time, reliability, increase in confidence, reusability, less human effort, reduction of cost and increase in fault detection. The challenges include failure to achieve expected goals, difficulty in maintenance of test automation, Test automation needs more time to mature, false expectations and lack of skilled people for test automation tools. From web survey, it is observed that almost all the benefits and challenges are prevalent in industry. The benefits such as fault detection and confidence are in contrary to the results of SLR. The challenge about the appropriate test automation strategy has 24 % disagreement from the respondents and 30% uncertainty. The reason is that the automation strategy is totally dependent on the test manager of the project. When asked “Does automated software testing fully replace manual testing”, 80% disagree with this challenge. Conclusion: The classification of the AST studies using systematic mapping gives an overview of the work done in the area of AST and also helps to find research coverage in the area of AST. These results can be used by researchers to use the gaps found in the mapping studies to carry on future work. The results of SLR and web survey clearly show that the practitioners clearly realize the benefits and challenges of AST reported in the literature. / Mobile no: +46723069909

Context: Exploratory Testing (ET) and Scripted Testing (ST) are two of the more commonly practiced manual testing approaches in industry. ST is a traditional testing approach in which testing is carried out by executing pre-designed test cases. While in ET, learning, test designing and test execution are carried out simultaneously. In many instances, ET and ST complement each other very well in projects; however, proponents of ET claim that ET is more cost-beneficial in comparison to ST. Moreover, a few studies have indicated that ET is more effective in defect detection. Nevertheless, to the best of our knowledge, no study has been conducted to compare the costs and benefits of these two approaches. Objectives: The aim of this study was to conduct a qualitative Cost-Benefit Analysis (CBA) of ET in comparison with ST. By comparing and analyzing these two testing approaches, this study attempts to aid in decision-making with respect to how resources should be allocated for ET and ST for certain projects. Methods: The factors of costs and benefits of ET and ST were identified by conducting six semi-structured interviews in industry. Based on the analysis of these factors, a CBA model is proposed. The academic and industrial evaluation of the proposed CBA model was performed by conducting five interviews with researchers and practitioners. In addition, a qualitative CBA of a process of ET, Session-Based Testing Management (SBTM), and a process of ST, Test-Case Based Testing (TCBT) is conducted by collecting data through questionnaires and interviews with industry practitioners. A total of 22 questionnaire responses and seven interviews were analyzed. Results: By analyzing the identified cost and benefit factors, a CBA model was developed based on the testing phases stated in the ISO/IEC 29119 standard. A qualitative CBA of the SBTM process in comparison with the TCBT process was conducted by applying the CBA model in a questionnaire. The following findings were gathered from the CBA: • The differences of the SBTM and TCBT processes are identified by an analysis of the activities performed in various organizations, which the respondents belonged to. • The results of the analysis and comparison, of the costs (in terms of effort) and benefits (quality of the testing activities) of these two testing processes, are presented with respect to each testing phase. • The factors that impact the costs and benefits of using SBTM and TCBT, are summarized and discussed in this report. • The scenarios, in which SBTM and TCBT can be more cost-beneficial, are identified based on practitioners’ opinions. Conclusions: According to the survey results, industry practitioners consider SBTM as more cost-beneficial in comparison with TCBT, particularly in the test design, implementation and test execution phases. However, industry practitioners also stressed that ET should not be considered as a replacement for ST. In some contexts, testing objectives are better achieved through a more scripted approach, while, in other contexts, testing objectives will benefit more from the ability to create and improve tests as they are being executed. Whether a testing approach is valuable or cost-beneficial also depends on the context of project and the required benefits.

Software testing is considered as one of the key phases in the software-development life cycle (SDLC). The main objective of software testing is to detect the faults either through manual testing or with automated testing approach. The most commonly adopted software testing approach in industries is test case based testing (TCBT) which is usually done manually. TCBT is mainly used by the software testers to formalize and guide their testing activities and set theoretical principals for testing. On the other hand, model based testing (MBT) is widely used automation software testing technique to generate and execute the tests. Both techniques are showing their prominence in real time with some pros and cons. However, there is no formal comparison available between these two techniques. The main objective of this thesis work is to find out the difference in test cases in TCBT and MBT in terms of providing better test coverage ( Statement, Branch and Path), requirement traceability, cost and time. To fulfill the aims of the research we have conducted interviews for static validation, and later we did an experiment for validating those results dynamically. The analysis of experiment results showed that the requirement traceability in MBT generated test cases are very hard to make the test cases traceable to the requirements, particularly with the open-source tool Model J-Unit. However, this can be done by using other commercial tools like Microsoft Spec Explorer or Conformiq Qtronic. Furthermore, we found by conducting experiment, that MBT consumes less time thus it is cost-effective as compared to TCBT and also MBT show better test coverage than TCBT. Moreover, we found that, in our case, requirement traceability is better in traditional TCBT approach as compared to MBT. / +4746851975

Developing software without considering the potential changes it mighthave to undergo in the future can be a costly mistake. This is becauseits maintenance costs can become very expensive as they can consumeover 90% of the total life-cycle costs. Incorporating maintainability insoftware has for this reason become highly attractive since it can significantlyreduce the maintenance costs and therefore save companies andsoftware developers a fortune. This thesis presents a software tool thathas been developed to aid Bombardier in the verification of computerbasedinterlocking (CBI) systems. The tool automatically generates testcases which represent the different tests that verify the interlocking system.The paper is divided into two parts. The first part focuses on themaintainability of the tool while the second part investigates whetherthe tool can speed up the testing process of CBI-systems at Bombardier.The results show that the tool is highly maintainable and that tests onCBI-systems can be performed significantly faster with it. / Att utveckla mjukvara utan hänsyn till de eventuella förändringar somden måste genomgå i framtiden kan bli ett kostsamt misstag. Dettaberor på att dess underhållskostnader kan bli mycket dyra eftersom dekan konsumera över 90% av de totala kostnaderna. Det har därför blivitväldigt attraktivt att lägga fokus på underhållbarhet när man utvecklarmjukvara eftersom det kan avsevärt minska underhållskostnadernaoch därmed spara mjukvaruutvecklare och företag en förmögenhet. Idenna uppsats presenteras ett verktyg som har utvecklats för att hjälpaBombardier med verifiering av datorställverk. Verktyget automatisktgenererar testfall som representerar de olika testen som verifierar ställverket.Uppsatsen är uppdelad i två delar. Den första delen fokuserarpå verktygets underhållbarhet medan den andra delen undersöker omverktyget kan påskynda testprocessen för datorställverk på Bombardier.Resultaten visar att verktyget är väldigt underhållbart och att

Large and complex software that is frequently changed leads to testing challenges. It is well established that the later a fault is detected in software development, the more it costs to fix. This thesis aims to research and develop a method of generating relevant and non-redundant test cases for a regression test suite, to catch bugs as early in the development process as possible. The research was executed at Axis Communications AB with their products and systems in mind. The approach utilizes user data to dynamically generate a Markov chain model and with a Markov chain Monte Carlo method, strengthen that model. The model generates test case proposals, detects test gaps, and identifies redundant test cases based on the user data and data from a test suite. The sampling in the Markov chain Monte Carlo method can be modified to bias the model for test coverage or relevancy. The model is generated generically and can therefore be implemented in other API-driven systems. The model was designed with scalability in mind and further implementations can be made to increase the complexity and further specialize the model for individual needs.