In this thesis, I/O signal recognizers, called VIRTUALBLOCKS, are synthesized to interface with a SYSTEM UNDER TEST (SUT). Methods for automated synthesis of virtualblocks allow us to simulate environment interfaces with SUT and also perform fault detection on SUT. Such methods must be able to recognize incoming sequences of signals from SUT, and upon the signal recognition determine the proper outgoing sequences of signals to SUT. We characterize our systems into four distinctive systems: system under test, AUXILIARY SYSTEM, controller and external environment. The auxiliary system is represented as a form of condition system Petri net (virtualblocks) and interacts with SUT along with the interaction among the controller and the external environment. Fault detection is performed by subsystems called DETECTBLOCKS synthesized from the virtualblocks. We present construction procedures for virtualblocks andamp; detectblocks and discuss the notion of LEGALITY and DETECTABILITY. Finally, we illustrate our approach using a model of a scanner control unit.

The growing demand for UI test automation has triggered the development of many tools. Researchers and developers have been continuously working to further improvise the existing approaches. If we look at GUI test evolution we can observe a clear progress from manual testing towards complete automation. Numerous approaches have been made to automate the GUI testing process. Record and playback tools, key-word driven methodologies, event flow exploration strategies, model based approaches are continuously evolving with higher level of automation. Similarly, new ideas and strategies to make these tests efficient are also emerging. Optimization of this resource consuming activity is another very important aspect in this area.  Dependencies between different tests can create deadlock scenarios, while running larger test suites. A concept of Ordered Test Suite can be used to cope with such dependencies. Following the Model Driven Architecture initiative by Object Management Group, a new global trend of Model Driven Engineering is creating a big sensation in the field of model based software development. Using the same principle, studies have also been made to automatically generate tests from models. Behavioral models can be made using the model driven approaches and these models can be analyzed to generate tests automatically. This master thesis addresses different approaches made for Graphical User Interface test automation, some optimization issues and solutions, a case study done at a software company to automate User Interface testing and a model driven approach for automatic test case generation.

Functional Test of any given system is used to verify that the top level system is performing according to the specifications and all of the sub systems, i.e., hardware, software, inputs, outputs and sensors are operating properly. The term System is used here in context of any device or equipment consisting of hardware, software, sensors, virtual sensors and inputs / outputs. One of the examples of such a system is a semiconductor fabrication equipment. There have only been a few approaches that are used to perform the functional test of a system. Engineers typically develop custom test equipment to interface with the system under test and determine proper functioning of all the sub systems and behavior of overall system. In another method, domain experts, utilize the software of the system under test, and manually actuate / stimulate sub systems and then observe and record outcomes to determine whether the system exhibited correct behavior. A novel solution of a reusable functional test automation framework is developed. The framework interfaces with the system under test via the exposed API, and allow domain experts with minimal or no programming background to create test suites to methodically test overall functionality of the system. / text

Multi-tenancy in relational databases has been a topic of interest for a couple of years. On the one hand, ever increasing capabilities and capacities of modern hardware easily allow for multiple database applications to share one system. On the other hand, cloud computing leads to outsourcing of many applications to service architectures, which in turn leads to offerings for relational databases in the cloud, as well. The ability to benchmark multi-tenancy database systems (MT-DBMSs) is imperative to evaluate and compare systems and helps to reveal otherwise unnoticed shortcomings. With several tenants sharing a MT-DBMS, a benchmark is considerably different compared to classic database benchmarks and calls for new benchmarking methods and performance metrics. Unfortunately, there is no single, well-accepted multi-tenancy benchmark for MT-DBMSs available and few efforts have been made regarding the methodology and general tooling of the process. We propose a method to benchmark MT-DBMSs and provide a framework for building such benchmarks. To support the cumbersome process of defining and generating tenants, loading and querying their data, and analyzing the results we propose and provide MULTE, an open-source framework that helps with all these steps.

Model-based testing (MBT) is a black-box software testing technique that focuses on specification of the system-under-test (SUT) and/or its environment. It uses models to automatically generate a large number of tests. To the best of our knowledge, no study has investigated the correlation of model coverage with SUT coverage using more advanced coverage criteria (such as edge-pair coverage) and the correlation of coverage (at model level and SUT level) with test suite effectiveness using non-adequate test suites in the context of MBT despite the prominence of non-adequate test suites in industry. To carry out the investigation, we extend an existing open-source MBT tool called Modbat to measure edge-pair coverage at model level, implement a new tool called PaCovForJbc to measure edge-pair coverage, edge coverage and node coverage at SUT level. Finally, we perform an experiment using these tools applied on three projects: “ArrayList”, and “LinkedList” of Java standard library, and “Apache ZooKeeper”. Overall, the results suggest the following: Edge and edge-pair coverage at model level often have a moderate to high correlation with the same type of coverage at SUT level, while that link between model and SUT for node coverage is weaker. Moreover, coverage criteria at SUT level often have a moderate to high correlation with test suite effectiveness, and a coverage criterion at SUT level has a slightly higher correlation with test suite effectiveness than the same type of coverage at model level. Regarding coverage at model level, edge and edge-pair coverage at model level have a slightly higher correlation with test suite effectiveness than node coverage at model level. Note that the mentioned suggestions need to be taken with discretion, because results vary depending on the project and/or coverage criterion under investigation. / Modellbaserad testning (MBT) är en black-box-testteknik som fokuserar på specifikation av system-under-test (SUT) och/eller dess miljö. MBT använder modeller för att generera ett stort antal tester automatiskt. Såvitt vi vet, finns ingen studie som undersökt korrelationen mellan modelltäckning och täckning av SUT med hjälp av mer avancerade täckningskriterier såsom kantparstäckning. Dessutom finns ingen studie som undersökt korrelationen mellan täckning (på modellnivå och SUT-nivå) och effektivitet av icke- adekvata testsviter som genereras med hjälp av MBT trots betydelsen av icke-adekvata testsviter i industrin. För att utföra undersökningen, utökar vi ett ”open-source” MBT-verktyg som kallas för Modbat för att mäta kantparstäckning på modellnivå. Dessutom implementerar vi ett nytt verktyg som kallas för PaCovForJbc för att mäta kantpars-, kant- och nodtäckning på SUT-nivå. Till slut utför vi experiment genom att applicera Modbat och PaCovForJbc på tre projekt: ”ArrayList” och ”LinkedList” av Javas standardbibliotek samt ”Apache ZooKeeper”. Sammantaget indikerar resultaten följande: Kant- och kantparstäckning på modellnivå har ofta en måttlig till hög korrelation med samma typ av täckning på SUT- nivå, medan länken mellan modell och SUT för nodtäckning är svagare. Dessutom har täckningskriterier på SUT-nivå ofta en måttlig till hög korrelation med testsvitseffektivitet, och ett täckningskriterium på SUT-nivå har en aning högre korrelation med testsvitseffektivitet än samma typ av täckning på modellnivå. Angående täckning på modellnivå har kant- och kantparstäckning på modellnivå en aning högre korrelation med testsvitseffektivitet än nodtäckning på modellnivå. Observera att de nämnda förslagen måste tas med diskretion, eftersom resultaten varierar beroende på projektet och/eller täckningskriteriet som undersöks.

I dagens informationssamhälle pågår en växande trend av molnbaserade tjänster. Vid implementering av molnbaserade tjänster används metoden Virtualisering. Denna metod minskar behovet av antal fysiska datorsystem i ett datacenter. Vilket har en positiv miljöpåverkan eftersom energikonsumtionen minskar när hårdvaruresurser kan utnyttjas till sin fulla kapacitet. Molnbaserade tjänster skapar samhällsnytta då nya aktörer utan teknisk bakgrundskunskap snabbt kan komma igång med verksamhetsberoende tjänster. För tillämpning av Virtualisering används en så kallad Hypervisor vars uppgift är att distribuera molnbaserade tjänster. Efter utvärdering av vetenskapliga studier har vi funnit att det finns skillnader i prestanda och funktionalitet mellan olika Hypervisors. Därför väljer vi att göra en prestanda- samt funktionsanalys av Hypervisors som kommer från de största aktörerna på marknaden. Dessa är Microsoft Hyper-V Core Server 2012, Vmware ESXi 5.1.0 och Citrix XenServer 6.1.0 Free edition. Vår uppdragsgivare är försvarsmakten som bekräftade en stor efterfrågan av vår undersökning. Rapporten innefattar en teoretisk grund som beskriver tekniker bakom virtualisering och applicerbara användningsområden. Genomförandet består av två huvudsakliga metoder, en kvalitativ- respektive kvantitativ del. Grunden till den kvantitativa delen utgörs av ett standardsystem som fastställdes utifrån varje Hypervisors begränsningar. På detta standardsystem utfördes prestandatester i form av dataöverföringar med en serie automatiserade testverktyg. Syftet med testverktygen var att simulera datalaster som avsiktligt påverkade CPU och I/O för att avgöra vilka prestandaskillnader som förekommer mellan Hypervisors. Den kvalitativa undersökningen omfattade en utredning av funktionaliteter och begränsningar som varje Hypervisor tillämpar. Med tillämpning av empirisk analys av de kvantitativa mätresultaten kunde vi fastställa orsaken bakom varje Hypervisors prestanda. Resultaten visade att det fanns en korrelation mellan hur väl en Hypervisor presterat och vilken typ av dataöverföring som den utsätts för. Den Hypervisor som uppvisade goda prestandaresultat i samtliga dataöverföringar är ESXi. Resultaten av den kvalitativa undersökningen visade att den Hypervisor som offererade mest funktionalitet och minst begränsningar är Hyper-V. Slutsatsen blev att ett mindre datacenter som inte planerar en expansion bör lämpligtvis välja ESXi. Ett större datacenter som både har behov av funktioner som gynnar molnbaserade tjänster och mer hårdvaruresurser bör välja Hyper-V vid implementation av molntjänster. / A growing trend of cloud-based services can be witnessed in todays information society. To implement cloud-based services a method called virtualization is used. This method reduces the need of physical computer systems in a datacenter and facilitates a sustainable environmental and economical development. Cloud-based services create societal benefits by allowing new operators to quickly launch business-dependent services. Virtualization is applied by a so-called Hypervisor whose task is to distribute cloud-based services. After evaluation of existing scientific studies, we have found that there exists a discernible difference in performance and functionality between different varieties of Hypervisors. We have chosen to perform a functional and performance analysis of Hypervisors from the manufacturers with the largest market share. These are Microsoft Hyper-V Core Server 2012, Vmware ESXi 5.1.0 and Citrix XenServer 6.1.0 Free edition. Our client, the Swedish armed forces, have expressed a great need of the research which we have conducted. The thesis consists of a theoretical base which describes techniques behind virtualization and its applicable fields. Implementation comprises of two main methods, a qualitative and a quantitative research. The basis of the quantitative investigation consists of a standard test system which has been defined by the limitations of each Hypervisor. The system was used for a series of performance tests, where data transfers were initiated and sampled by automated testing tools. The purpose of the testing tools was to simulate workloads which deliberately affected CPU and I/O to determine the performance differences between Hypervisors. The qualitative method comprised of an assessment of functionalities and limitations for each Hypervisor. By using empirical analysis of the quantitative measurements we were able to determine the cause of each Hypervisors performance. The results revealed that there was a correlation between Hypervisor performance and the specific data transfer it was exposed to. The Hypervisor which exhibited good performance results in all data transfers was ESXi. The findings in the qualitative research revealed that the Hypervisor which offered the most functionality and least amount of constraints was Hyper-V. The conclusion of the overall results uncovered that ESXi is most suitable for smaller datacenters which do not intend to expand their operations. However a larger datacenter which is in need of cloud service oriented functionalities and requires greater hardware resources should choose Hyper-V at implementation of cloud-based services.