Context. With the rapid development of mobile devices, mobile applications have occupied an increasingly large market. Software testing is an important factor in ensuring software quality [59]. In order to develop high-quality mobile applications, software companies are increasingly paying attention to mobile application testing. We found that there are relatively few studies on mobile application testing methods and tools in the real industry, and this direction of research has been very helpful for some software companies' mobile application testing, so we chose this direction for exploratory research. Objectives. The purpose of our research is to investigate the software company's field of mobile application testing. We mainly study the methods and tools of mobile application testing that are now popular in enterprises, the challenges faced by mobile application testing in software companies and how they solve challenges. Method. We used two methods to answer our research questions. We chose literature review and survey as research methods. The literature review method gives us an insight into the areas of expertise and mobile testing. Survey is used to answer our research questions. Our data collection method is the questionnaire survey method. When we completed the data collection, we performed statistics and analysis on the data and answered our research questions. Result. Through the literature review method, we summarize the methods and tools for mobile application testing in the literature. By sending 46 surveys to different software companies, we analyzed and analyzed the methods and tools for mobile application testing that are popular in the enterprise and the reasons for choosing these methods. We also listed the company's mobile application testing. Challenges and solutions encountered. Conclusions. We answered all the research questions. We analyzed the feedback from 46 questionnaires and got information about mobile application testing. We read a lot of literature and summarized the relevant conclusions in the literature. Our findings can help people in related fields, and we can provide them with 46 data and analysis results. We can also provide mobile application methods and tools that are popular among readers in the enterprise.

This work introduces three new coverage criteria based on combinatorial-based event and element sequences that occur in the mobile environment. The novel combinatorial-based criteria are used to reduce, prioritize, and generate test suites for mobile applications. The combinatorial-based criteria include unique coverage of events and elements with different respects to ordering. For instance, consider the coverage of a pair of events, e1 and e2. The least strict criterion, Combinatorial Coverage (CCov), counts the combination of these two events in a test case without respect to the order in which the events occur. That is, the combination (e1, e2) is the same as (e2, e1). The second criterion, Sequence-Based Combinatorial Coverage (SCov), considers the order of occurrence within a test case. Sequences (e1, ..., e2) and (e2,..., e1) are different sequences. The third and strictest criterion is Consecutive-Sequence Combinatorial Coverage (CSCov), which counts adjacent sequences of consecutive pairs. The sequence (e1, e2) is only counted if e1 immediately occurs before e2. The first contribution uses the novel combinatorial-based criteria for the purpose of test suite reduction. Empirical studies reveal that the criteria, when used with event sequences and sequences of size t=2, reduce the test suites by 22.8%-61.3% while the reduced test suites provide 98.8% to 100% fault finding effectiveness. Empirical studies in Android also reveal that the event sequence criteria of size t=2 reduce the test suites by 24.67%-66% while losing at most 0.39% code coverage. When the criteria are used with element sequences and sequences of size t=2, the test suites are reduced by 40\% to 72.67%, losing less than 0.87% code coverage. The second contribution of this work applies the combinatorial-based criteria for test suite prioritization of mobile application test suites. The results of an empirical study show that the prioritization criteria that use element and event sequences cover the test suite's elements, events, and code faster than random orderings. On average the prioritized orderings cover all elements within 21.81% of the test suite, all events within 45.99% of the test suite, and all code within 51.21% of the test suite. Random orderings achieve full code coverage with 84.8% of the test suite on average. The third contribution uses the combinatorial-based criteria for test suite generation. This work modifies the random walk tool used from prior experiments to give weight (preference) to coverage of the combinatorial-based event and element criteria. The use of Element SCov and CSCov criteria result in test suites that increase code coverage for three of the four subject applications. Specifically, the code coverage increases by 0.29%-5.89% with SCov and 1.36%-6.79% with CSCov in comparison to the original random walk algorithm. The SCov criterion increases total sequence coverage by 5%-88% and the CSCov criterion increases sequence coverage by 13%-68%. One criteria, Element CCov, failed to increase code coverage for two of the four applications. The contributions of this dissertation show that the novel combinatorial-based criteria using sequences of events and elements offer improvements to different testing strategies for mobile applications, including test suite reduction, prioritization, and generation.

Användandet av mobila applikationer har växt radikalt de senaste åren och de samverkar med många system. Därför ställs det högre krav på kvaliteten och att applikationen ska anpassas till många olika enheter, operativsystem samt plattformar. Detta gör att test av mobila applikationer blivit viktigare och större. Detta arbete har bedrivits som en jämförande fallstudie inom området test av mobila applikationer samt testverktyg. Syftet har varit att beskriva hur testning av mobila applikationer sker idag vilket gjorts genom litteraturstudier och intervjuer med IT-företag. Ett annat syfte har varit att utvärdera fyra testverktyg, deras för- och nackdelar samt hur de kan användas vid testning av mobila applikationer och jämföras mot manuell testning utan testverktyg. Detta har gjorts genom att skapa förstahandserfarenheter baserat på användandet av testverktygen. Under arbetet har vi utgått från mobila applikationer som vi fått tillgång till av Triona, som varit vår samarbetspartner.Idag finns många olika testverktyg som kan användas som stöd för testningen men få företag har implementerat något eftersom det kräver både tid och kompetens samt valet av testverktyg kan vara svårt. Testverktygen har olika för- och nackdelar vilket gör att de passar olika bra beroende på typ av projekt och applikation. Fördelar med att använda testverktyg är möjligheten att kunna automatisera, testa på flera enheter samtidigt samt få tillgång till enheter via molnet. Utmaningarna är att det kan vara svårt att installera och lära sig testverktyget samt att licenserna kan vara dyra. Det är därför viktigt att redan innan implementationen veta vilka tester och applikationer testverktygen ska användas till samt vem som ska använda det. Utifrån vår studie kan slutsatsen dras att inget testverktyg är helt komplett men de kan bidra med olika funktioner vilket effektiviserar delar av testningen av mobila applikationer. / The use of mobile applications has grown dramatically in recent years and they interact with many systems. Therefore the requirement on the quality is higher and the application must often be adapted to many devices, operating systems and platforms. This means the testing of mobile applications becomes more and more important. This study has been conducted as a comparative case study in the field testing of mobile applications and testing tools. The aim has been to describe how the testing of mobile applications takes place today which have been made through literature studies and interviews with IT-companies. Another aim was to evaluate four testing tools, their advantages and disadvantages and how they can be used in the testing of mobile applications and compared to manual testing without testing tools. This was done by using and experience the testing tools. During the study, we have used mobile applications that we got access from Triona, who has been our partner.Today there are many different testing tools that can be used to support testing, but few companies have implemented these because it requires both time and expertise as well as the selection of testing tools can be difficult. Testing tools have different advantages and disadvantages which make them fit differently depending on the type of project and application. Advantages of using testing tools is the ability to automate, test on multiple devices simultaneously and get access to physical devices via the cloud. The challenges are that it can be difficult to install and learn the testing tool and that the licenses can be expensive. It is therefore important that even before implementation to know which tests and applications the testing tools will be used in and who will use it. Based on our study we can conclude that no testing tools are complete but they may contribute with different features which streamlines parts of the mobile applications testing.

Automated test generation for Andriod apps with reinforcement learning algorithms often produce test suites with redundant coverage. We looked at minimizing test suites that have already been generated based on state–action–reward–state–action (SARSA) algorithms. In this dissertation, we hypothesize that there is room for improvement by introducing novel hybrid approaches that combine SARSA-generated test suites with greedy reduction algorithms following the principle of Head-up Guidance System (HGS™) approach. In addition, we apply an empirical study on Android test suites that reveals the value of these new hybrid methods. Our novel approaches focus on post-processing test suites by applying greedy reduction algorithms. To reduce Android test suites, we utilize different coverage criteria including event-based criterion (EBC), element-based criterion (ELBC), and combinatorial-based sequences criteria (CBSC) that follow the principle of combinatorial testing to generate sequences of events and elements. The proposed criteria effectively decreased the test suites generated by SARSA and revealed a high performance in maintaining code coverage. These findings suggest that test suite reduction using these criteria is particularly well suited for SARSA-generated test suites of Android apps.

Mobile applications play an important role in the dissemination of computing and information resources. They are often used in domains such as mobile banking, e-commerce, and health monitoring. Cost-effective testing techniques in these domains are critical. This dissertation contributes novel techniques for automatic construction of mobile application test suites. In particular, this work provides solutions that focus on the prohibitively large number of possible event sequences that must be sampled in GUI-based mobile applications. This work makes three major contributions: (1) an automated GUI testing tool, Autodroid, that implements a novel online approach to automatic construction of Android application test suites (2) probabilistic and combinatorial-based algorithms that systematically sample the input space of Android applications to generate test suites with GUI/context events and (3) empirical studies to evaluate the cost-effectiveness of our techniques on real-world Android applications. Our experiments show that our techniques achieve better code coverage and event coverage compared to random test generation. We demonstrate that our techniques are useful for automatic construction of Android application test suites in the absence of source code and preexisting abstract models of an Application Under Test (AUT). The insights derived from our empirical studies provide guidance to researchers and practitioners involved in the development of automated GUI testing tools for Android applications.

The number of mobile applications has increased tremendously over the last decade, thereby also having increased the importance of mobile application testing. Testing is a very crucial process in the development of a mobile application to ensure reliability and proper functionality. In an Agile/DevOps environment, test automation is an integral part in order to support the continuous integration and continuous delivery/deployment principle. There is a plethora of mobile application testing automation frameworks available in the market today. Android testing frameworks were chosen for this study because of the prevalence of Android applications in the market today. The focus of this study is the comparison of the most popular mobile application testing automation frameworks in terms of performance and capabilities in an Agile/DevOps environment. In order to achieve this, firstly, the three most popular mobile application testing automation frameworks to be used for the study were identified. Secondly, the Key Performance Indicators and capabilities to be used for comparison of the frameworks were identified. Then, test cases in a complex mobile application were designed for analysing the frameworks based on the criteria gathered. Finally, the test cases were integrated in a Continuous Integration/ Continuous Delivery pipeline to gather results and perform a more detailed comparison in an Agile/DevOps environment. From this study, Espresso turned out to be the best framework in terms of KPI analysed beating the other frameworks in all the KPIs, while Robotium was second best followed by Appium. But on the other hand, Appium was the best framework in terms of capabilities by having ability to execute the maximum number of capabilities analysed followed by Robotium while Espresso was the worst framework in this criteria. / Antalet mobilapplikationer har ökat enormt under det senaste decenniet och därmed har även vikten av mobilapplikationstest ökat. Testning är en mycket viktig process i utvecklingen av en mobilapplikation för att säkerställa tillförlitlighet och korrekt funktionalitet. I en Agile/DevOps miljö är testautomation en viktig del för att stödja principen av kontinuerlig integration och kontinuerlig leverans/driftsättning. Det finns en mängd ramverk för automatiserad testning av mobilapplikationer på marknaden idag. Testramverk för Android valdes för denna studie på grund av utbredningen av Android-applikationer på marknaden idag. Fokus för denna studie är jämförelsen av de mest populära automatiserade testramverk för mobilapplikationer med avseende på prestanda och lämplighet i en Agile/DevOps miljö. För att uppnå detta, identifierades först de tre mest populära automatiserade testramverk för mobilapplikationer som skulle användas för studien. Sedan identifierades de viktigaste resultatindikatorerna och kapaciteterna som skulle användas för att jämföra ramverken. Därefter designades testfall i en komplex mobilapplikation för att analysera ramverken baserat på de bestämda kriterierna. Slutligen integrerades testfallen i en pipeline för kontinuerlig integration /kontinuerlig leverans för att samla resultat och utföra en mer detaljerad jämförelse i en Agile/DevOps miljö. Från denna studie visade sig Espresso vara det bästa ramverket när det gäller KPI-analys, som slog de andra ramarna i alla KPI: er, medan Robotium var näst bäst följt av Appium. Men å andra sidan var Appium det bästa ramverket med avseende på kapacitet. Den hade förmågan att utföra det maximala antalet kapacitet som analyserades, följt av Robotium medan Espresso var det sämsta ramverket i dessa kriterier.