Maintaining quality in software is a difficult task for several reasons, such as, company growth, time-to-market demands, code complexity and more. GUI testing tools and Continuous Integration (CI) are common practice today to tackle some of the issues with maintaining software quality. However, these techniques bring a set of challenges. Visual Regression Testing (VRT) is a special kind of GUI testing technique focused upon image-based assertions. This study presents an implementation and investigation of the benefits and drawbacks of introducing VRT for a CI environment in an industrial context. Additionally, the thesis investigates factors that need to be considered upon this transition. The results show that benefits are associated with this approach, such as, quicker feedback times and an increase in testing frequency. However, drawbacks and implications were also identified, such as, test maintenance and organizational concerns, indicating careful consideration needs to be taken by an organization before proceeding with an implementation.

No description available.

The effectiveness of GUI-based automated tests is a topic that is often discussed in the testing community. As software systems and GUIs are getting more advanced, the testing of these systems may sometimes be both time-consuming and repetitive. Therefore, automated tests are used more frequently to create more agile working and to decrease the number of manual tests. There is an ongoing discussion about the benefits and shortcomings of implementing automated GUI test compared to performing manual GUI tests. Increasingly, companies today are implementing automated GUI tests, yet relatively few are analyzing enough when choosing their test strategy. The purpose of this thesis was to explore different test strategies in a system as complex as Nasdaq’s clearing system NRTC. The first strategy used manual GUI tests, while the second strategy used a test tool called Selenium WebDriver to automate the GUI tests. Hopefully, this research will contribute to the already accumulated knowledge about automated GUI tests and potentially help in choosing the right testing strategy. The findings of this research showed that the implementation time of manual tests was much faster, and that implementing automated test would be timeconsuming. The reason for this is that the GUI was not very testable, making it difficult to create the test scripts. In addition, the was a difference in runtime when using both strategies. The automated tests only took a couple of seconds to execute, compared to the manual tests that took a couple of minutes. After both strategies was investigated, an analysis was done by looking at the effectivity, maintainability, robustness, and the overall results to be able to find benefits and shortcomings. In conclusion, GUI automation cannot replace manual test completely. By not including the human capabilities, the testing process becomes very limited.

The validation approach for assuring quality of software does often include the conduction of tests. Software testing includes a wide range of methodology depending on the system level and the component under test. Graphical user interface (GUI) testing consists of high level tests that assert that functions and design element in user interfaces work as expected. The research conducted in this paper focused on GUI testing of web based applications and the movement towards automated testing within the software industry. The question which formed the basis for the study was the following: How should a generic test automation framework be designed in order to allow maintenance between developers and non-developers? The study was conducted on a Swedish consultant company that provides e-commerce web solutions. A work strategy approach for automated testing was identified and an automation framework prototype was produced. The framework was evaluated through a pilot study where testers participated through the creation of a test suite for a specific GUI testing area. Time estimations were collected as well as qualitative measurements through a follow up survey. This paper presents a work strategy proposal for automated tests together with description of the framework system design. The results are presented with a subsequent discussion about the benefits and complexity of creating and introducing automated tests within large scale systems. Future work suggestions are also addressed together with accountancy of the frameworks usefulness for other testing areas besides GUI testing.

Context:Contemporary software development is a socio-technical activity requiring extensive collaboration among individuals with diverse expertise. Software testing is an integral part of software development that also depends on various expertise. GUI-based testing allows assessing a system’s GUI and its behavior through its graphical user interface. Collaborative practices in software development, like code reviews, not only improve software quality but also promote knowledge exchange within teams. Similar benefits could be extended to other areas of software engineering, such as GUI-based testing. However, collaborative practices for GUI-based testing necessitate a unique approach since general software development practices, perceivably, can not be directly transferred to software testing. Goal:This thesis contributes towards a tool-supported approach enabling collaborative GUI-based testing. Our distinct goals are (1) to identify processes and guidelines to enable collaboration on GUI-based testing artifacts and (2) to operationalize tool support to aid this collaboration. Method:We conducted a systematic literature review identifying code review guidelines for GUI-based testing. Further, we conducted a controlled experiment to assess the efficiency and potential usability issues of Augmented Testing. Results:We provided guidelines for reviewing GUI-based testing artifacts, which aid contributors and reviewers during code reviews. We further provide empirical evidence that Augmented Testing is not only an efficient approach to GUI-based testing but also usable for non-technical users, making it a promising subject for further research in collaborative GUI-based testing. Conclusion:Code review guidelines aid collaboration through discussions, and a suitable testing approach can serve as a platform to operationalize collaboration. Collaborative GUI-based testing has the potential to improve the efficiency and effectiveness of such testing.

No description available.

Naujuose programiniuose produktuose plačiai taikoma mobilioji įranga (mobilieji telefonai, delniniai kompiuteriai). Programinės įrangos kūrimo procese svarbią vietą užima testavimas. Dabartinių mobiliųjų įranginių apribojimai (darbo sparta, atminties kiekis, energija, ekrano dydis, platformų įvairumas) kelia naujas problemas programinės įrangos kūrimo procesui, tame tarpe ir testavimui. Testavimo proceso automatizavimas leidžia sumažinti bandymų trukmę, padidinti testavimo darbų apimtį. Programinės įrangos testavime mobiliems įrenginiams svarbu išskirti grafinės vartotojo sąsajos testavimą. Svarbi testavimo dalis – testavimas pagal specifikaciją. Šiame darbe nagrinėjamas grafinės vartotojo sąsajos automatizuotas testavimas mobiliai įrangai, nustatomi tikslai, galimi keliai jiems pasiekti bei galimas testavimo aplinkos praplėtimas testų generavimui pagal specifikaciją. / Mobile devices such as cell phones and personal digital assistance are widely used in new software products. Testing takes important place in software development process. Constraints of mobile devices (speed, amount of memory, energy, small screen, wide range of platforms) raise new problems for software development process including testing phase. Automated approach of software testing reduces testing time and increases testing range. It is important to distinguish graphic user interface as a special part of testing. The important part of testing is specification based testing. The aim of these master theses is to analyze automated testing of GUI for mobile devices, define testing tasks and enhance mobile device testing framework by providing means for test case generation from specification.

Företag arbetar ofta efter hårda krav från kunder där lösningar måste levereras på ett tidseffektivt sätt och samtidigt hålla en hög kvalitet. Detta i form av felfria och robusta system vilket delvis kan åstadkommas med hjälp av testning. Kraven på snabb leverans och hög kvalitet är däremot motpoler till varandra; snabb leverans genomförs ofta på bekostnad av kvalitet och tvärtom. Agila arbetsmetoder med tidiga och frekventa leveranser har ändrat på detta, men kräver samtidigt en ständig kvalitetsförsäkring under arbetets löptid. Under utveckling av mjukvara enligt dessa metoder förekommer därför en kontinuerlig kvalitetssäkringsprocess för att säkerställa att produkten dels uppfyller vad kravspecifikationen avser samt att levererad produkt håller en hög tekniskt kvalitet i form av buggfri och robust kod som innehar stor pålitlighet för framtiden. Då manuell testning är en kostnad- och resurskrävande metod har automatiserad testning blivit ett aktuellt alternativ för ökad effektivitet och hållbar utveckling. Målet med det här arbetet har varit att för Precio Systemutveckling AB utreda möjligheterna för en implementation av automatiserad testning av användargränssnitt, i en hos företaget redan existerande och etablerad utvecklingsprocess. Arbetet har genomförts med en inledande förstudie om testning med fokus på varför det är en extra viktig faktor i dagsläget. Detta följs av ett avsnitt där existerande teori och teknik för testning i generell mening avhandlats, följt av en närmare studie på hur automatiserad testning är rekommenderad att utföras ur ett perspektiv från utveckling av produkter inom Microsoft-teknologi. / Software companies work under a strict pressure from customers where solutions must be delivered in a timely manner as well as providing high quality and value. The products should be robust and without errors, which partially can be accomplished by testing during the development process. The requirement for an early delivery and a high quality does not always come hand in hand, but with the increased use of agile software development methods, this can be achieved. During agile development of software, there is a continuous process to ensure the quality of what is being developed, both to make sure that all the functional requirements are fulfilled, but also that the code behind is robust and dependable for the future. Since manual testing can be both time and resource consuming, automated testing has become a modern alternative to increase productivity and to maintain a sustainable development process. The goal of this thesis work has been to investigate the possibilities of implementing a solution for automated UI testing in an already existing development process at the company Precio Systemutveckling AB. The work has been conducted in three steps, starting with a literature study about testing in general, followed by an extensive research into suitable tools and technology for testing that exists today. After this, a deeper look was made at what the recommended solutions for implementing automated testing in a Microsoft-oriented enviroment were. The work was concluded with an actual implementation of automated testing on premise at Precio.

This dissertation introduces a hybrid strategy for automated testing of Android applications that combines reinforcement learning and test suite prioritization. These approaches aim to improve the effectiveness of the testing process by employing reinforcement learning algorithms, namely Q-learning and SARSA (State-Action-Reward-State-Action), for automated test case generation. The studies provide compelling evidence that reinforcement learning techniques hold great potential in generating test cases that consistently achieve high code coverage; however, the generated test cases may not always be in the optimal order. In this study, novel test case prioritization methods are developed, leveraging pairwise event interactions coverage, application state coverage, and application activity coverage, so as to optimize the rates of code coverage specifically for SARSA-generated test cases. Additionally, test suite prioritization techniques are introduced based on UI element coverage, test case cost, and test case complexity to further enhance the ordering of SARSA-generated test cases. Empirical investigations demonstrate that applying the proposed test suite prioritization techniques to the test suites generated by the reinforcement learning algorithm SARSA improved the rates of code coverage over original orderings and random orderings of test cases.

Mobile applications play an important role in the dissemination of computing and information resources. They are often used in domains such as mobile banking, e-commerce, and health monitoring. Cost-effective testing techniques in these domains are critical. This dissertation contributes novel techniques for automatic construction of mobile application test suites. In particular, this work provides solutions that focus on the prohibitively large number of possible event sequences that must be sampled in GUI-based mobile applications. This work makes three major contributions: (1) an automated GUI testing tool, Autodroid, that implements a novel online approach to automatic construction of Android application test suites (2) probabilistic and combinatorial-based algorithms that systematically sample the input space of Android applications to generate test suites with GUI/context events and (3) empirical studies to evaluate the cost-effectiveness of our techniques on real-world Android applications. Our experiments show that our techniques achieve better code coverage and event coverage compared to random test generation. We demonstrate that our techniques are useful for automatic construction of Android application test suites in the absence of source code and preexisting abstract models of an Application Under Test (AUT). The insights derived from our empirical studies provide guidance to researchers and practitioners involved in the development of automated GUI testing tools for Android applications.