GUI testing is expensive to perform manually. Software systems involving a heterogeneous set of components exclude the applicability of specific GUI testing techniques. Visual GUI Testing (VGT) is a test automation technique that combines image recognition with scripts. It applies to almost any GUI driven application. VGT is proven to be cost-effective in comparison with manual testing. Still, it is expensive to maintain. This study investigates if test script design by following specific guidelines positively affects maintenance costs. A case study was conducted to identify best practices for VGT w.r.t. maintenance time. Four VGT versions were developed for each manual test case. These consisted of two design versions, with/without guidelines, for the two VGT-tools EyeAutomate and Sikuli. Data was collected using time measurements, observations, and interviews. Results highlighted differences in initial development time and maintenance time between the two design versions. In total, 44 observations were collected. 17 were related to the design versions, 17 to the VGT-tools, and 10 to VGT in general, initial development, and the system under test. The interviews collected the perceptions of VGT in general, maintenance of the different VGT versions, and guidelines. In conclusion, the combination of the guidelines did not have a positive effect on maintenance in terms of costs and experience. However, some of the individual guidelines did. A rationale why the guidelines did not give the desired result was identified. Future research is necessary to investigate other combinations of guidelines, such as those identified as beneficial.

No description available.

Background. With the development and popularization of mobile Internet, smartphones are becoming more and more popular. Android is one of the most popular platforms of smartphones.  And application is one of the most important part of a smartphone. There are a lot of money and human resources spent on Android application development every year. And quiet a big part of them goes to quality assurance of applications. Graphic user interface (GUI) testing is one important part of its quality assurance. Android phones use touch screen as the major I/O method. Therefore, GUI testing on android platform shall be different to conventional software applications that are designed to run on desktop environment. Objectives. The aim of this research is to assess the performance of two GUI testing approaches (2nd vs 3rd generation) of automated UI testing in terms of testing Android applications. By assessing these approaches, we could hopefully get insights of their advantages and limitations for using them in the context of Android development. And this aim can be divided into three objectives, to compare the time spent on implementing test cases of each tool, to compare the time costed when executing test cases of each tool, to compare the number of defects found by each tool. Methods. The research methodology we chose is controlled experiment. We have chosen UI Automator and Appium to represent 2nd generation GUI testing approach, EyeAutomate and SikuliX to represent 3rd generation GUI testing approach. We used each tool to implement and execute 120 test cases to compare them on the time spent on implementing test cases of each tool, the time costed when executing test cases of each tool, the number of real defects found by each tool, and the number of false positives found by each tool. Results. Tools using 3rd generation GUI testing approach take less time to implement test cases than tools using 2nd generation GUI testing approach. And there is no specific pattern when comparing tools using 2nd and 3rd generation GUI testing approaches in terms of time cost on executing test cases. It is different between different test cases. Besides false positive alerts appear at a much higher frequency in tools using 3rd generation GUI testing approach than tools using 2nd generation GUI testing approach. While, real defects found by each tool are the same. Conclusions. 3rd generation GUI testing approach is more efficient in terms of implementing test cases than 2nd generation GUI testing approach. But 3rd generation GUI testing approach finds much more false positives than 2nd generation approach. To decide if a defect alert is false positive or not requires human effort. In a long term, it may accumulate huge lost on human efforts. Therefore, to maintain test cases, 3rd generation approach consumes lots of human efforts.