Context. GUI-based automation testing is a costly and tedious activity in practice. As GUIs are well-known for being modified and redesigned throughout the development process, the corresponding test scripts are not valid anymore thereby being a hindrance to automation. Hence, substantial effort is invested in maintaining GUI test scripts which often leads to rework or waste due to improper decisions. As a result, practitioners have identified the need for decision support regarding when should GUI automation testing begin and how to make it easier and also identify what are the factors leading to waste in GUI-based automation testing. The current literature provides solutions relating to automation in general and few answers for GUI based-automation testing. Such generic answers might not be applicable to GUI test automation and also industries new to GUI development and testing. Thus, it is necessary to validate if the general solutions are applicable to GUI test automation and find additional answers that are not identified previously from practitioners opinions in an industrial context. Objectives. Capture relevant information regarding the current approach for GUI test automation within the subsystems from a case company. Next, identify the criteria for when to begin automation, testability requirements and factors associated with waste from literature and practice. Methods. We conducted a multiple-case study to explore opinions of practitioners in two subsystems at a Swedish telecommunication industry implementing GUI-automation testing. We conducted a literature review to identify answers from scientific literature prior to performing a case study.A two-phased interview was performed with different employees to collect their subjective opinions and also gather their opinions on the evidence collected from the literature. Later, Bayesian synthesis method was used to combine subjective opinions of practitioners with research-based evidence to produce context-specific results. Results. We identified 12 criteria for when to begin automation, 16 testability requirements and 15 factors associated with waste in GUI test automation.Each of them is classified into categories namely SUT-related,test-process related, test-tool related, human and organizational, environment and cross-cutting. New answers which were not present in the existing literature in the domain of the research are found. Conclusions. On validating the answers found in literature, it was revealed that the answers applicable for software test automation, in general, are valid for GUI automation testing as well. Since we incorporated subjective opinions to produce context specific results, we gained an understanding that every practitioner has their own way of working. Hence, this study aids in developing a common understanding to support informed subjective decisions based on evidence.

In order to develop and secure the functionality of its cellular communications systems, Ericsson deals with numerous R&D and I&V activities. One important aspect is interoperability with mobile terminals from different vendors on the world market. Therefore Ericsson co-operates with mobile platform and user equipment manufacturers. These companies visit the interoperability developmental testing (IoDT) laboratories in Linköping to test their developmental products and prototypes in order to certify compliance with Ericsson's products. The knowledge exchange is mutual, Ericsson as well as the user equipment manufacturers benefit from the co-operation. The goal of this master's thesis performed at Ericsson AB is to suggest areas in which the IoDT testing can be automated in order to minimize time consuming and tedious work tasks. Primarily the search should be aimed at replacing manual tasks in use today. The thesis suggests a number of IoDT tasks that might be subject for automation. Among these one is chosen for implementation. The thesis also includes an implementation part. The task that has been chosen for implementation is the network verification after base station controller software upgrade procedure. This is not a core IoDT function but it implies a lot of work, and is often performed. The automation project is also supposed to act as a springboard for future automation within IoDT. The forthcoming LTE standard will require a lot of IoDT testing, and therefore the automation capabilities should be investigated. The thesis shows that automation work is possible, and that the startup process is straightforward. Existing tools are easy to use, and well supported. The network verification automated test scope has been successful.

Ipendo Systems in Linköping, Sweden, is a small IT consulting firm developing among other things custom portal applications based on Microsoft Sharepoint 2010. The purpose of this thesis is to investigate whether the test tools TestComplete and LoadComplete provide sufficient compatibility for functional and non-functional testing of Ipendo Systems’ Sharepoint based applications, as well as design testing related activities to be incorporated into their existing software development framework. A test process maturity framework was chosen and applied for the design of the testing activities, while the test tool investigation resulted in guidelines on how to apply the tools effectively and circumvent any issues discovered.

Mobile devices have evolved rapidly in recent years and increased usage and performance are pushing contemporary battery technology to its limits. The constrained battery resources mean that the importance of energy-efficient application design is growing and in this regard wireless network accesses are a major contributor to a mobile device's overall energy consumption. Additionally, the energy consumption characteristics of modern cellular technologies make small volumes of poorly scheduled traffic account for a substantial share of a device's total energy consumption. However, quantifying the communication energy footprint is cumbersome, making it difficult for developers to profile applications from an energy consumption perspective and optimize traffic patterns. This thesis examines the traffic patterns of the Android client of the popular multimedia streaming service Spotify with the intention to reduce its energy footprint, in terms of 3G energy consumption. The application's automated test environment is extended to capture network traffic, which is used to estimate energy consumption. Automated system tests are designed and executed on a physical Android device connected to a 3G network, shedding light on the traffic patterns of different application features. All traffic between the Spotify client application and the backend servers is encrypted. To extract information about the traffic, the application code is instrumented to output supplementary information to the Android system log. The system log is then used as a source of information to attribute data traffic to different application modules and specific lines of code. Two simple traffic shaping techniques, traffic aggregation and piggybacking, are implemented in the application to provide more energy-efficient traffic patterns. As a result, 3G energy consumption during normal music playback is reduced by 22-54%, and a more contrived scenario achieves a 60% reduction. The reductions are attained by rescheduling a small class of messages, most notably data tracking application usage. These messages were found to account for a small fraction of total traffic volume, but a large portion of the application's overall 3G energy consumption.

The number of web applications continues to grow rapidly. Many applications that were traditionally desktop-based such as word processors, email clients or even file managers have now their web alternatives that often have bigger user base. While the software world is shifting from providing desktop-based applications to offering cloud-based services, challenges arise to provide high quality software in this radically different environment. Web-based applications are different in many aspects to their desktop equivalents. Web-based and desktop-based applications have radically dissimilar architecture, user interface, execution environment, requirements and limitations. To mention a few specific differences: the web application is executed in the context of a web browser, not operating system; code is executed in two separate places: on the server and in the web browser; code for the server is written in a different language than code for the client. Because of these differences, the traditional software testing techniques are only of limited use. To properly test a web application one needs to take into account numerous differences in Web-Browsers such as Internet Explorer, Mozilla Firefox, Google Chrome or Apple Safari. Simple input-output testing is useful only to limited degree, because output cannot be just compared to the predefined value but must be compared within the context of all the environments and operating systems the application supports. The fact that an application works in one browser does not guarantee that it will work at all in other browsers. The entire complexity of the problem makes testing web applications a very time consuming and difficult task. Often it takes significantly longer time to test the application than it is to implement it. Because of these difficulties, web applications are often tested exclusively manually by humans, and only to a limited degree. New techniques and technologies are necessary to approach the problem of web application testing. This paper includes a case study carried out during author’s work on a web-based application. It examines the current state-of-the-art of web application testing and suggests useful techniques in tackling the problem.

Diploma thesis deals with the implementation of automated tests to the process of testing web applications. Introduction of the theoretical part is devoted to familiarizing the reader with the concept of testing and illustration of its importance not only in software development. Author then defines the term, introduces the principles, and attention is paid to the interpretation of terms which are commonly used in testing. The next section contains specific division of this process in terms of of the used techniques, methods and levels of testing. By description, and stating the fundamental differences between manual and automated testing creates conditions for understanding the issues, on which work is focused. At the end of theoretical part there are characterized basic models of software development life cycle, in which the process of testing has its place. In the practical part the author at first draws attention to introduction of of web application in which automated tests will be implemented. Subsequently, based on the introduced methodology, he analyzes the process of testing this application. He defines the reasons for the implementation of automated testing, he marks the areas of coveragecreates working team and the timetable for implementation. On the basis of established criteria justifies the choice of tools to be used in automation. In the next chapter he accompanies reader systematically across the entire process of creation and implementation of automated tests, and intentionally complements it with practical examples. End is devoted to assessment of the results that have been achieved by the implementation of automated tests. By comparing them with manual tests author determines the effectiveness of automation. Calculating the estimated savings is determined by the return on investment in automated tests.

This thesis discusses the design and implementation of integrated penetration testing system. In the first two chapters, the reader is introduced to the topic of penetration testing. The basic techniques and classification of tests are described as well as some of the most widely used methodologies. It also discusses the need to automate the testing process. The fifth and sixth chapter discuss specification and detailed design of integrated penetration testing tool. Its implementation and the problems that had arisen during the process are the theme of chapter seven. The last part of the thesis describes practical experiments done with the tool and gives the reader some advice on securing computer networks.

Maintaining quality in software is a difficult task for several reasons, such as, company growth, time-to-market demands, code complexity and more. GUI testing tools and Continuous Integration (CI) are common practice today to tackle some of the issues with maintaining software quality. However, these techniques bring a set of challenges. Visual Regression Testing (VRT) is a special kind of GUI testing technique focused upon image-based assertions. This study presents an implementation and investigation of the benefits and drawbacks of introducing VRT for a CI environment in an industrial context. Additionally, the thesis investigates factors that need to be considered upon this transition. The results show that benefits are associated with this approach, such as, quicker feedback times and an increase in testing frequency. However, drawbacks and implications were also identified, such as, test maintenance and organizational concerns, indicating careful consideration needs to be taken by an organization before proceeding with an implementation.

The procedure of testing the implemented software is important and should be an essential and integrated part of the development process. In order for the testing to be meaningful it is important that the testing procedure ensures that the developed software meet certain requirements. The testing procure is often controlled by some sort of test specification. For many companies it is desirable to automate this procure. The focus of this thesis has been to automate a small subpart of the manual tests today performed related to SAAB:s air traffic management system. The automation has been achieved by studying the existing test specification which involves a lot of manual operations and to write software that mimics a few of these test cases. The thesis has resulted in a test framework which automates a small subset of the manual tests performed today. The framework has been designed to be scalable and to easily allow more test cases to be added by the personnel when time permits. The test framework has also been integrated with SAAB:s existing CI/CD workflow.

This project was designed to help Hi3G Access AB’s IT department to get a better overview of the testing state in the different cross functional teams. Furthermore it was designed to identify teams and system where regression tests had the potential and the need of being automated. This information was gathered by conducting a survey designed for all the software testers at the company, followed by performing semi-structured interviews with one tester from each team. For the most suitable system and test case, an automation solution prototype was created with an automation tool using an agile test automation process model. The automation tool was evaluated regarding complexity, userfriendliness, available documentation and performance. The team and the system decided to be most prioritized and suitable was the team that worked with the Enterprise resource planning (ERP) system. The automation tool used for the prototype was Oracle applications testing suite (OATS). The project revealed that many teams at Hi3G Access AB were quite far in the automation process. The tool evaluated in the project presented itself to be straightforward and easy to learn/use. However the tool was missing a widespread discussion board and also lacked free support. / Projektet var utformat för att hjälpa Hi3G Access ABs IT-avdelning få en bättre överblick gällande de olika korsfunktionella gruppers testsituation. Vidare var målet att identifiera team och system där regressionstester hade potential och behov av automatisering. Denna information samlades in genom att utföra en enkät avsedd för alla mjukvarutestare på företaget, följt av att utföra semistrukturerade intervjuer med en testare från varje team.För det mest lämpliga system och testfall skapades en automatiserad prototyplösning genom en agil testautomatiseringsmodell. Automatiseringsverktyget utvärderas gällande komplexitet, användarvänlighet, tillgänglig dokumentation och prestanda.Teamet och systemet som valdes som mest prioriterat och lämpligt var teamet som arbetade med Enterprise resource planning (ERP) systemet. Automatiseringsverktyget som användes för prototypen var Oracle Applications Testing Suite (OATS).Projektet visade att många team på Hi3G Access AB hade kommit ganska långt i automatiseringsprocessen. Verktyget som utvärderades visade sig vara okomplicerat och lätt att både lära sig och att använda. Däremot så var avsaknaden av ett utbrett diskussionsforum ett faktum samt att produkten saknade fri support.