No / Cloud computing is increasingly attracting large attention both in academic research and in industrial initiatives. However, despite the popularity, there is a lack of research on the suitability of software tools and parameters for creating and deploying Cloud test beds. Virtualization and how to set up virtual environments can be done through software tools, which are available as open source, but there still needs to be work in terms of which tools to use and how to monitor parameters with the suitability of hardware resources available. This paper discusses the concepts of virtualization, as a practical view point, presenting an in-depth critical analysis of open source cloud implementation tools such as CloudStack, Eucalyptus, Nimbus, OpenStack, OpenNebula, OpenIoT, to name a few. This paper analyzes the various toolkits, parameters of these tools, and their usability for researchers looking to deploy their own Cloud test beds. The paper also extends further in developing an experimental case study of using OpenStack to construct and deploy a test bed using current resources available in the labs at the University of Bradford. This paper contributes to the theme of software setups and open source issues for developing Cloud test bed for deploying and constructing private Cloud test bed.

Submitted by Lucienne Costa (lucienneferreira@ufcg.edu.br) on 2018-08-30T18:21:31Z No. of bitstreams: 1 IURY GREGORY MELO FERREIRA – DISSERTAÇÃO (PPGCC) 2017.pdf: 3496497 bytes, checksum: b497c83bd5c1b1ab2be30ab67272f5cd (MD5) / Made available in DSpace on 2018-08-30T18:21:31Z (GMT). No. of bitstreams: 1 IURY GREGORY MELO FERREIRA – DISSERTAÇÃO (PPGCC) 2017.pdf: 3496497 bytes, checksum: b497c83bd5c1b1ab2be30ab67272f5cd (MD5) Previous issue date: 2017-12-18 / Em ambientes de Computação na Nuvem, principalmente os que utilizam o modelo de infraestrutura como um serviço, a característica de elasticidade no provisionamento de recursos traz consigo a necessidade de gerenciar os recursos físicos de forma apropriada para preservar a qualidade de serviço aos seus usuários, e o bom desempenho da infraestrutura. Este trabalho propõe heurísticas que são capazes de auxiliar no balanceamento de carga dos servidores em uma infraestrutura de nuvem, propondo migrações para diminuir a sobrecarga nos servidores que foram identificados como sobrecarregados,visto que, como passar do tempo há uma variação natural na quantidade de recursos em uso. Esta variação é uma consequência da remoção ou adição de aplicações, ou até mesmo de tentativas de melhoramento do desempenho das aplicações através do provisionamento vertical. Uma ferramenta foi implementada para fazer uso dos algoritmos das heurísticas e assim auxiliar nos experimentos para a validação das mesmas. As métricas utilizadas vem diretamente de servidores heterogêneos da nuvem OpenStack do Laboratório de Sistemas Distribuídos. Os resultados obtidos mostram que além da diminuição no consumo de CPU dos servidores dos quais que estavam sobrecarregados, também é possível melhorar o desempenho destes servidores em alguns casos. / In CloudComputingenvironments,especiallythoseusingtheinfrastructureasaservice model, theelasticitycharacteristicinresourceprovisioningcomeswiththeneedtomanage resources sothequalityofservicecancontinuetobeguaranteedtousersandalsoto maintain agoodperformanceoftheinfrastructure.Thisworkproposesheuristicsthat are abletoassistintheloadbalancingoftheserversinaCloudinfrastructure,proposing migrations toreducetheoverheadintheserversthatwereidentifiedasoverloaded,since with thepassageoftimethereisanaturalvariationintheamountofresourcesinuse.This variationinaconsequenceofremovaloradditionofapplicationsandevenoftheusageof verticalscalingtoimproveapplication’sperformance.Atoolwasimplementedtomake use oftheheuristicalgorithmsandthustoaidintheexperimentsandtheirvalidation,the metrics usedcomedirectlyfromheterogeneousserversoftheOpenStackCloudofthe DistributedSystemsLaboratory.TheresultsshowthatinadditiontothedecreaseinCPU consumption ofserversthatwereoverloaded,itisalsopossibletoimprovetheperformance of theseserversinsomecases.

This thesis deals with implementation of continuous integration for build system Copr. The implementation uses framework Citool and its modules, which are already used for continuous integration of build system Koji. The outcome system can run the tests for the new package from the build system Copr and test it on virtual machine. This thesis shows way how to implement continuous integration for build system Copr.

With the advent of Smart Cities, Intelligent Transport System (ITS) has become an efficient way of offering an accessible, safe, and sustainable transportation system. Utilizing advances in Information and Communication Technology (ICT), ITS can maximize the capacity of existing transportation system without building new infrastructure. However, in spite of these technical feasibilities and significant performance-cost ratios, the deployment of ITS is limited in the real world because of several challenges associated with its architectural design. This thesis studies how to design a highly flexible and deployable architecture for ITS, which can utilize the recent technologies such as - cloud computing and the publish/subscribe communication model. In particular, our aim is to offer an ITS infrastructure which provides the opportunity for transport authorities to allocate on-demand computing resources through virtualization technology, and supports a wide range of ITS applications. We propose to use an Infrastructure as a Service (IaaS) model to host large-scale ITS applications for transport authorities in the cloud, which reduces infrastructure cost, improves management flexibility and also ensures better resource utilization. Moreover, we use a publish/subscribe system as a building block for developing a low latency ITS application, which is a promising technology for designing scalable and distributed applications within the ITS domain. Although cloud-based architectures provide the flexibility of adding, removing or moving ITS services within the underlying physical infrastructure, it may be difficult to provide the required quality of service (QoS) which decrease application productivity and customer satisfaction, leading to revenue losses. Therefore, we investigate the impact of service mobility on related QoS in the cloud-based infrastructure. We investigate different strategies to improve performance of a low latency ITS application during service mobility such as utilizing multiple paths to spread network traffic, or deploying recent queue management schemes. Evaluation results from a private cloud testbed using OpenStack show that our proposed architecture is suitable for hosting ITS applications which have stringent performance requirements in terms of scalability, QoS and latency. / Baksidestext: Intelligent Transport System (ITS) can utilize advances in Information and Communication Technology (ICT) and maximize the capacity of existing transportation systems without building new infrastructure. However, in spite of these technical feasibilities and significant performance-cost ratios, the deployment of ITS is limited in the real world because of several challenges associated with its architectural design.  This thesis studies how to design an efficient deployable architecture for ITS, which can utilize the advantages of cloud computing and the publish/subscribe communication model. In particular, our aim is to offer an ITS infrastructure which provides the opportunity for transport authorities to allocate on-demand computing resources through virtualization technology, and supports a wide range of ITS applications. We propose to use an Infrastructure as a Service (IaaS) model to host large-scale ITS applications, and to use a publish/subscribe system as a building block for developing a low latency ITS application. We investigate different strategies to improve performance of an ITS application during service mobility such as utilizing multiple paths to spread network traffic, or deploying recent queue management schemes. / <p>Artikel 4 Network Centric Performance Improvement for Live VM Migration finns i avhandlingen som manuskript. Nu publicerat konferenspaper. </p>

CERN Scalable Analytics Section currently offers shared YARN clusters to its users as monitoring, security and experiment operations. YARN clusters with data in HDFS are difficult to provision, complex to manage and resize. This imposes new data and operational challenges to satisfy future physics data processing requirements. As of 2018, there were over 250 PB of physics data stored in CERN’s mass storage called EOS. Hadoop-XRootD Connector allows to read over network data stored in CERN EOS. CERN’s on-premise private cloud based on OpenStack allows to provision on-demand compute resources. Emergence of technologies as Containers-as-a-Service in Openstack Magnum and support for Kubernetes as native resource scheduler for Apache Spark, give opportunity to increase workflow reproducability on different compute infrastructures with use of containers, reduce operational effort of maintaining computing cluster and increase resource utilization via cloud elastic resource provisioning. This trades-off the operational features with datalocality known from traditional systems as Spark/YARN with data in HDFS.In the proposed architecture of cloud-managed Spark/Kubernetes with data stored in external storage systems as EOS, Ceph S3 or Kafka, physicists and other CERN communities can on-demand spawn and resize Spark/Kubernetes cluster, having fine-grained control of Spark Applications. This work focuses on Kubernetes CRD Operator for idiomatically defining and running Apache Spark applications on Kubernetes, with automated scheduling and on-failure resubmission of long-running applications. Spark Operator was introduced with design principle to allow Spark on Kubernetes to be easy to deploy, scale and maintain with similar usability of Spark/YARN.The analysis of concerns related to non-cluster local persistent storage and memory handling has been performed. The architecture scalability has been evaluated on the use case of sustained workload as physics data reduction, with files in ROOT format being stored in CERN mass-storage called EOS. The series of microbenchmarks has been performed to evaluate the architecture properties compared to state-of-the-art Spark/YARN cluster at CERN. Finally, Spark on Kubernetes workload use-cases have been classified, and possible bottlenecks and requirements identified. / CERN Scalable Analytics Section erbjuder för närvarande delade YARN-kluster till sina användare och för övervakning, säkerhet, experimentoperationer, samt till andra grupper som är intresserade av att bearbeta data med hjälp av Big Data-tekniker. Dock är YARNkluster med data i HDFS svåra att tillhandahålla, samt komplexa att hantera och ändra storlek på. Detta innebär nya data och operativa utmaningar för att uppfylla krav på dataprocessering för petabyte-skalning av fysikdata.Från och med 2018 fanns över 250 PB fysikdata lagrade i CERNs masslagring, kallad EOS. CERNs privata moln, baserat på OpenStack, gör det möjligt att tillhandahålla beräkningsresurser på begäran. Uppkomsten av teknik som Containers-as-a-Service i Openstack Magnum och stöd för Kubernetes som inbyggd resursschemaläggare för Apache Spark, ger möjlighet att öka arbetsflödesreproducerbarheten på olika databaser med användning av containers, minska operativa ansträngningar för att upprätthålla datakluster, öka resursutnyttjande via elasiska resurser, samt tillhandahålla delning av resurser mellan olika typer av arbetsbelastningar med kvoter och namnrymder.I den föreslagna arkitekturen av molnstyrda Spark / Kubernetes med data lagrade i externa lagringssystem som EOS, Ceph S3 eller Kafka, kan fysiker och andra CERN-samhällen på begäran skapa och ändra storlek på Spark / Kubernetes-klustrer med finkorrigerad kontroll över Spark Applikationer. Detta arbete fokuserar på Kubernetes CRD Operator för idiomatiskt definierande och körning av Apache Spark-applikationer på Kubernetes, med automatiserad schemaläggning och felåterkoppling av långvariga applikationer. Spark Operator introducerades med designprincipen att tillåta Spark över Kubernetes att vara enkel att distribuera, skala och underhålla. Analys av problem relaterade till icke-lokal kluster persistent lagring och minneshantering har utförts. Arkitekturen har utvärderats med användning av fysikdatareduktion, med filer i ROOT-format som lagras i CERNs masslagringsystem som kallas EOS. En serie av mikrobenchmarks har utförts för att utvärdera arkitekturegenskaperna såsom prestanda jämfört med toppmoderna Spark / YARN-kluster vid CERN, och skalbarhet för långvariga dataprocesseringsjobb.

Context. As the number of organizations using cloud application platforms to host their applications increases, the priority of distributing physical resources within those platforms is increasing simultaneously. The goal is to host a higher quantity of applications per physical server, while at the same time retain a satisfying rate of performance combined with certain scalability. The modern needs of customers occasionally also imply an assurance of certain privacy for their applications. Objectives. In this study two types of instances for hosting applications in cloud application platforms, virtual machines and application containers, are comparatively analyzed. This investigation has the goal to expose advantages and disadvantages between the instances in order to determine which is more appropriate for being used in cloud application platforms, in terms of performance, scalability and user isolation. Methods. The comparison is done on a server running Linux Ubuntu 16.04. The virtual machine is created using Devstack, a development environment of Openstack, while the application container is hosted by Docker. Each instance is running an apache web server for handling HTTP requests. The comparison is done by using different benchmark tools for different key usage scenarios and simultaneously observing the resource usage in respective instance. Results. The results are produced by investigating the user isolation and resource occupation of respective instance, by examining the file system, active process handling and resource allocation after creation. Benchmark tools are executed locally on respective instance, for a performance comparison of the usage of physical resources. The amount of CPU operations executed within a given time is measured in order determine the processor performance, while the speed of read and write operations to the main memory is measured in order to determine the RAM performance. A file is also transmitted between host server and application in order to compare the network performance between respective instance, by examining the transfer speed of the file. Lastly a set of benchmark tools are executed on the host server to measure the HTTP server request handling performance and scalability of each instance. The amount of requests handled per second is observed, but also the resource usage for the request handling at an increasing rate of served requests and clients. Conclusions. The virtual machine is a better choice for applications where privacy is a higher priority, due to the complete isolation and abstraction from the rest of the physical server. Virtual machines perform better in handling a higher quantity of requests per second, while application containers is faster in transferring files through network. The container requires a significantly lower amount of resources than the virtual machine in order to run and execute tasks, such as responding to HTTP requests. When it comes to scalability the prefered type of instance depends on the priority of key usage scenarios. Virtual machines have quicker response time for HTTP requests but application containers occupy less physical resources, which makes it logically possible to run a higher quantity of containers than virtual machines simultaneously on the same physical server.

The goal of this diploma thesis is to design the user interface and implement the information system as a web application. Using the custom implemented library the system communicates with GraphQL server which manages the client data. The thesis describes possible solutions for physical servers automatization. The application provides the application interface to manage virtual servers. Automatization is possible without human interaction. Connection to the virtualization technologies is handled by web interface APIs or custom scripts running in the virtual system terminal. There is a monitoring system built over project components. The thesis also describes the continuous integration using Gitlab tools. Running the configuration task is solved using the Unix CRON system.