Efficiency as a term in software testing is, in the research community, a term that is not so well defined. In the industry, and specifically the test tool industry, it has become a sales pitch without meaning. GUI testing in its manual form is a time consuming task, which can be thought of as repetitive and tedious by testers. Using human testers to perform a task, where focus is hard to keep, often ends in defects going unnoticed. The purpose of this thesis is to collect knowledge on the area efficiency in software testing, but focusing more on efficiency in GUI testing in order to keep the scope focused. Part of the purpose is also to test the hypothesis that automated GUI testing is more efficient than traditional, manual GUI testing. In order to reach the purpose, the choice fell to use case study research as the main research method. Through the case study, a theoretical study was performed to gain knowledge on the subject. To gain data used for an analysis in the case study, the choice fell on using a semi-experimental research approach where one automated GUI testing technique called Capture & Replay was tested against a more traditional approach towards GUI testing. The results obtained throughout the case study gives a definition on efficiency in software testing, as well as three measurements on efficiency, those being defect detection, repeatability of test cases, and time spent with human interaction. The result also includes the findings from the semi-experimental research approach where the testing tools Squish, and TestComplete, where used beside a manual testing approach. The main conclusion deducted in this work is that an automated approach towards GUI testing can become more efficient than a manual approach, in the long run. This is when efficiency is determined on the points of defect detection, repeatability, and time.

Ipendo Systems in Linköping, Sweden, is a small IT consulting firm developing among other things custom portal applications based on Microsoft Sharepoint 2010. The purpose of this thesis is to investigate whether the test tools TestComplete and LoadComplete provide sufficient compatibility for functional and non-functional testing of Ipendo Systems’ Sharepoint based applications, as well as design testing related activities to be incorporated into their existing software development framework. A test process maturity framework was chosen and applied for the design of the testing activities, while the test tool investigation resulted in guidelines on how to apply the tools effectively and circumvent any issues discovered.

Automated UI testing tools are an essential part of the software development process. While comparative studies have been conducted to evaluate various tools, the identification of the challenges faced by today’s testers remains a crucial task. Our study aims to provide insight into this subject by comparing two widely used testing tools, Selenium and TestComplete, and determining their ease of use and time efficiency. We also consider the learning curve for new testers and provide guidelines for selecting the right tool. Our goal is to evaluate how Selenium and TestComplete compare in terms of ease of use and time efficiency. Our research questions are: (RQ1) Which tool has a shorter implementation time for developers, Selenium or TestComplete? (RQ2) Which tool has a faster execution time for test cases, Selenium or TestComplete? (RQ3) Which tool has a higher number of defects found during testing, Selenium or TestComplete? (RQ4) Which tool has a higher number of false positives found during testing, Selenium or TestComplete? To answer these questions, a controlled experiment was conducted, showing that both tools have similar implementation times. However, Selenium has a faster execution time than TestComplete and a significantly lower occurrence of false positives. These results suggest that Selenium provides a more reliable and user-friendly testing process. In terms of defects found, the data gathering was insufficient and future research is required. Furthermore, we found that there is a limitation to what elements and applications can be tested. In this regard, Selenium showed a more significant limitation than TestComplete for testing an online game application.

Software Quality is a key concern for any companies working with software development. This is true due to the fact that the success of any software directly depends on Quality of software. It is expected that the software is of best quality for a long duration of time. With the introduction of Mobile applications the task of maintaining the quality of an application has been difficult and have faced many challenges. Many companies working with mobile application have reformed their process in order to maintain the quality of their application. The introduction of Automation testing in the test process is one such reform that have changed the face of mobile application testing in today’s world. This work deals with the concepts of Automation System testing for the mobile application which is until now a new thing and it has many things yet to be explored. The approach to automation testing is simple yet unique for the department of PT-MT/Quality Management in Robert Bosch GmbH based in Leinfelden, Stuttgart. Over here a selection and implementation of a test framework will be done for Automation testing of the mobile Applications that are being developed. For this a requirement specification document is being created which will form the basis for selecting a framework from the KT Analysis table. Finally, a framework TestComplete will be implemented for the already developed application "PLR measure&go" The implementation will include all the procedure required to set up the test framework as a part of documentation. The framework TestComplete will be used to create System test for iOS and Android operation system. Lastly the execution of test and the Result reporting is being shown as a complete process for Automation testing.

<p>The products The Company manufactures are used in an area of the industry where errors leading to a stop in production can be quite expensive. Therefore are testing of the products important and the tests can also give indications about the quality of the products.</p><p>The Company is in a phase where they are developing a new product line to support all existing and future products. In this phase, it was decided that all products will use a common framework for unit testing and a common build system for all products. One part of the thesis was to investigate and evaluate different frameworks for unit testing and tools for a build system. The unit test framework that were evaluated are Cppunit, cfix, NUnit, Boost test library, Unit Testing++ and CxxTest. The result of the evaluation was that CppUnit were recommended. For the build system MSBuild, NAnt, Automated Build Studio and Cruise Control .NET were evaluated. The recommended tools for a build system is MSBuild in combination with Cruise Control .Net if The Company is interested in the functionality Cruise Control .Net has to offer.</p><p>The Company also has an interest in evaluating the current test processes and identify improvements as a part of The Company’ s objective were all products should follow a common test process. In order to identify these improvements a literature study of four test process improvement frameworks (Test Process Improvement, Test Maturity Model Integrated, Minimal Test Practice Framework and Test Improvement Model) were carried out. Out of these four frameworks Test Process Improvement (TPI) were chosen to assist when identifying improvements. With the help of TPI a limited assessment took place to give indications about the test maturity for three of The Company’s products where two of the products had low maturity. Results of the improvement measures can be summed up with the need to harmonize the documents, standardize and document the various processes.</p><p>As a last part of the thesis the possibility to automate testing of two of the products graphical user interfaces with the program Test Complete were investigated. For one of the products the result was that it worked satisfactorily and for the other product it did not work at all. This resulted in recommendations for how The Company should proceed with automated testing of the graphical user interface.</p> / <p>De produkter som företaget tillverkar används i huvudsak inom ett område av branschen där fel som leder till stopp i produktionen kan vara ganska dyrt. Detta gör testning av produkterna viktiga och tester kan också ge indikationer om kvaliteten på produkterna.</p><p>Företaget är i en fas där man utvecklar en ny produktlinje som ska stödja alla befintliga och framtida produkter. I denna fas har man beslutat att alla produkterna ska använda ett gemensant ramverk för enhetstestning och även använda ett gemensamt byggsystem för samtliga produkter. En del av examensarbetet var att undersöka och utvärdera olika ramverk för enhets testning och verktyg för byggsystem. De ramverk som utvärderades var CppUnit, cfix, NUnit, Boost test library, unitTest++ och CxxTest. Utvärderingen ledde fram till att CppUnit rekommenderades till företaget. Verktyg som utvärderades för byggsystem var MSBuild, NAnt, Automated Build Studio och Cruise Control .Net. För byggsystem rekommenderas MSBuild i kombination med Cruise Control .Net ifall företaget är intresserade av den extra funktionalitet som Cruise Control .Net har att erbjuda.</p><p>Företaget har även ett intresse av att utvärdera den nuvarande testprocessen och identifiera förbättringar som ett led i att befintliga produkter skall följa en gemensam testprocess. För att kunna identifiera dessa förbättringar utfördes en litteraturstudie över fyra stycken test process förbättrings ramverk (Test Process Improvent, Test Maturity Model integrated, Minimal Test Practice Framework och Test Improvement Model). Utav dessa fyra ramverk så valdes Test Process Improvement (TPI) ut som en hjälp för att identifiera förbättringar. Med hjälp av TPI utfördes en begränsad bedömning av företagets mogenhetsgrad på tre produkter, där två av produkterna har en låg mogenhetsgrad. Resultat av förbättringsåtgärderna kan sammanfattas med att man bör harmonisera dokument, standardisera och dokumentera olika processer.</p><p>Som en sista del i examensarbetet så utvärderades möjligheten att automatisera testning på två av produkternas grafiska användargränssnitt med programmet TestComplete. För en av produkterna blev resultatet att det fungerade tillfredställande och för den andra produkten så fungerade det inte alls. Resultatet blev rekommendationer för hur företaget borde gå vidare med automatisering av testnig på det grafiska användargränssnittet.</p>

As the complexity of software system increases, delivering quality software successfully becomes an ever more challenging task. Applying automated testing techniques effectively to the software development process can reduce software testing effort substantially and assure software quality cost-effectively. Thus the future of software testing will rely heavily on automated testing techniques. This thesis describes a practical approach to automated software testing by investigating and analyzing different automation test tools in real-world situations. In view of the fact that the key to successful automated testing is planning, understanding the requirements for automated testing and effectively planning is critical and essential. iv This thesis presents the design and implementation of an automated testing framework. It consists of an automated testing tool, which is based on the commercial product TestComplete, as well as associated testing processes. The application area is telephony communications software. To demonstrate the viability of our automated testing approach, we apply our testing framework to a Voice-over-IP telephony application called Desktop Assistant. This case study illustrates the benefits and limitations of our automated testing approach effectively.

The products The Company manufactures are used in an area of the industry where errors leading to a stop in production can be quite expensive. Therefore are testing of the products important and the tests can also give indications about the quality of the products. The Company is in a phase where they are developing a new product line to support all existing and future products. In this phase, it was decided that all products will use a common framework for unit testing and a common build system for all products. One part of the thesis was to investigate and evaluate different frameworks for unit testing and tools for a build system. The unit test framework that were evaluated are Cppunit, cfix, NUnit, Boost test library, Unit Testing++ and CxxTest. The result of the evaluation was that CppUnit were recommended. For the build system MSBuild, NAnt, Automated Build Studio and Cruise Control .NET were evaluated. The recommended tools for a build system is MSBuild in combination with Cruise Control .Net if The Company is interested in the functionality Cruise Control .Net has to offer. The Company also has an interest in evaluating the current test processes and identify improvements as a part of The Company’ s objective were all products should follow a common test process. In order to identify these improvements a literature study of four test process improvement frameworks (Test Process Improvement, Test Maturity Model Integrated, Minimal Test Practice Framework and Test Improvement Model) were carried out. Out of these four frameworks Test Process Improvement (TPI) were chosen to assist when identifying improvements. With the help of TPI a limited assessment took place to give indications about the test maturity for three of The Company’s products where two of the products had low maturity. Results of the improvement measures can be summed up with the need to harmonize the documents, standardize and document the various processes. As a last part of the thesis the possibility to automate testing of two of the products graphical user interfaces with the program Test Complete were investigated. For one of the products the result was that it worked satisfactorily and for the other product it did not work at all. This resulted in recommendations for how The Company should proceed with automated testing of the graphical user interface. / De produkter som företaget tillverkar används i huvudsak inom ett område av branschen där fel som leder till stopp i produktionen kan vara ganska dyrt. Detta gör testning av produkterna viktiga och tester kan också ge indikationer om kvaliteten på produkterna. Företaget är i en fas där man utvecklar en ny produktlinje som ska stödja alla befintliga och framtida produkter. I denna fas har man beslutat att alla produkterna ska använda ett gemensant ramverk för enhetstestning och även använda ett gemensamt byggsystem för samtliga produkter. En del av examensarbetet var att undersöka och utvärdera olika ramverk för enhets testning och verktyg för byggsystem. De ramverk som utvärderades var CppUnit, cfix, NUnit, Boost test library, unitTest++ och CxxTest. Utvärderingen ledde fram till att CppUnit rekommenderades till företaget. Verktyg som utvärderades för byggsystem var MSBuild, NAnt, Automated Build Studio och Cruise Control .Net. För byggsystem rekommenderas MSBuild i kombination med Cruise Control .Net ifall företaget är intresserade av den extra funktionalitet som Cruise Control .Net har att erbjuda. Företaget har även ett intresse av att utvärdera den nuvarande testprocessen och identifiera förbättringar som ett led i att befintliga produkter skall följa en gemensam testprocess. För att kunna identifiera dessa förbättringar utfördes en litteraturstudie över fyra stycken test process förbättrings ramverk (Test Process Improvent, Test Maturity Model integrated, Minimal Test Practice Framework och Test Improvement Model). Utav dessa fyra ramverk så valdes Test Process Improvement (TPI) ut som en hjälp för att identifiera förbättringar. Med hjälp av TPI utfördes en begränsad bedömning av företagets mogenhetsgrad på tre produkter, där två av produkterna har en låg mogenhetsgrad. Resultat av förbättringsåtgärderna kan sammanfattas med att man bör harmonisera dokument, standardisera och dokumentera olika processer. Som en sista del i examensarbetet så utvärderades möjligheten att automatisera testning på två av produkternas grafiska användargränssnitt med programmet TestComplete. För en av produkterna blev resultatet att det fungerade tillfredställande och för den andra produkten så fungerade det inte alls. Resultatet blev rekommendationer för hur företaget borde gå vidare med automatisering av testnig på det grafiska användargränssnittet.

Software Quality is a key concern for any companies working with software development. This is true due to the fact that the success of any software directly depends on Quality of software. It is expected that the software is of best quality for a long duration of time. With the introduction of Mobile applications the task of maintaining the quality of an application has been difficult and have faced many challenges. Many companies working with mobile application have reformed their process in order to maintain the quality of their application. The introduction of Automation testing in the test process is one such reform that have changed the face of mobile application testing in today’s world. This work deals with the concepts of Automation System testing for the mobile application which is until now a new thing and it has many things yet to be explored. The approach to automation testing is simple yet unique for the department of PT-MT/Quality Management in Robert Bosch GmbH based in Leinfelden, Stuttgart. Over here a selection and implementation of a test framework will be done for Automation testing of the mobile Applications that are being developed. For this a requirement specification document is being created which will form the basis for selecting a framework from the KT Analysis table. Finally, a framework TestComplete will be implemented for the already developed application "PLR measure&go" The implementation will include all the procedure required to set up the test framework as a part of documentation. The framework TestComplete will be used to create System test for iOS and Android operation system. Lastly the execution of test and the Result reporting is being shown as a complete process for Automation testing.