The examiner of cloud computing systems in the last few years observes that there is a trend of the emergence of new Big Data frameworks every single year. Since Hadoop was developed in 2007, new frameworks followed it such as Spark, Storm, Heron, Apex, Flink, Samza, Kafka ... etc. Each framework is developed in a certain way to target and achieve certain objectives better than other frameworks do. However, there are few common functionalities and aspects that are shared between these frameworks. One vital aspect all these frameworks strive to achieve is better reliability and faster recovery time in case of failures. Despite all the advances in making datacenters dependable, failures actually still happen. This is particularly onerous for long-running “big data” applications, where partial failures can lead to significant losses and lengthy recomputations. This is also crucial for streaming systems where events are processed and monitored online in real time, and any delay in data delivery will cause a major inconvenience to the users.<div>Another observation is that some reliability implementations are redundant between different frameworks. Big data processing frameworks like Hadoop MapReduce include fault tolerance mechanisms, but these are commonly targeted at specific system/failure models, and are often redundant between frameworks. Encapsulating these implementations into one layer and making it shared between different applications will benefit more than one frame-work without the burden of re-implementing the same reliability approach in each single framework.<br></div><div>These observations motivated us to solve the problem by presenting two systems: Guardian and Warden. Guardian is tailored towards batch processing big data systems while Warden is targeted towards stream processing systems. Both systems are robust, RMS based, generic, multi-framework, flexible, customizable, low overhead systems that allow their users to run their applications with individually configurable fault tolerance granularity and degree, with only minor changes to their implementation.<br></div><div>Most reliability approaches carry out one rigid fault tolerance technique targeted towards one system at a time. It is more challenging to provide a reliability approach that is pluggable in multiple Big Data frameworks at a time and can achieve low overheads comparable with single targeted framework approaches, yet is flexible and customizable by its users to make it tailored towards their objectives. The genericity is attained by providing an interface that can be used in different applications from different frameworks in any part of the application code. The low overhead is achieved by providing faster application finish times with and without failures. The customizability is fulfilled by providing the users the options to choose between two fault tolerance guarantees (Crash Failures / Byzantine Failures) and, in case of streaming systems; it is combined with two delivery semantics (Exactly Once / At Most Once).<br></div><div>In other words, this thesis proposes the paradigm of dependable resources: big data processing frameworks are typically built on top of resource management systems (RMSs),and proposing fault tolerance support at the level of such an RMS yields generic fault tolerance mechanisms, which can be provided with low overhead by leveraging constraints on resources.<br></div><div>To the best of our knowledge, such approach was never tried on multiple big data batch processing and streaming frameworks before.<br></div><div>We demonstrate the benefits of Guardian by evaluating some batch processing frame-works such as Hadoop, Tez, Spark and Pig on a prototype of Guardian running on Amazon-EC2, improving completion time by around 68% in the presence of failures, while maintaining around 6% overhead. We’ve also built a prototype of Warden on the Flink and Samza (with Kafka) streaming frameworks. Our evaluations on Warden highlight the effectiveness of our approach in the presence of failures and without failures compared to other fault tolerance techniques (such as checkpointing)<br></div>

<p> First, NoSQL data-stores provide a set of features that is demanded by high perfor?mance computing (HPC) applications such as scalability, availability and schema flexibility. High performance computing (HPC) applications, such as metagenomics and other big data systems, need to store and analyze huge volumes of semi-structured data. Such applica?tions often rely on NoSQL-based datastores, and optimizing these databases is a challenging endeavor, with over 50 configuration parameters in Cassandra alone. As the application executes, database workloads can change rapidly over time (e.g. from read-heavy to write-heavy), and a system tuned for one phase of the workload becomes suboptimal when the workload changes. </p>

I takt med att världen digitaliseras och att allt fler företag börjar upptäcka fördelarna med molnbaserade IT-system, så misslyckas fortfarande hela 2/3 av alla IT-projekt trots att liknande projekt har gjorts många gånger förut. Denna kandidatuppsats har som syfte identifiera, beskriva och förklara de faktorer som påverkar en övergång från dagens lokalt lagrade system till molnbaserade IT-system, ur kund-, projektgrupp- och systemimplementatörsperspektiv. Primära empiriska data har samlats in genom kvalitativa intervjuer av fyra respondenter. Intervju-erna fokuserade på deras erfarenheter och uppfattningar om vilka faktorer som påverkar en övergång från lokalt lagrade system till molnbaserade IT-system. Respondenterna har olika perspektiv och har olika arbetsbakgrund, men gemensamt är att samtliga arbetar inom IT-området. Tre av dem har syste-mimplementatörsbakgrund och den fjärde arbetar som IT-chef hos ett kundföretag. En viktig slutsats från denna studie är att det inte finns lika stora risker vid en övergång till ett molnbaserat IT-system som marknaden upplever. De faktorer som har en hög påverkan på en fram-gångsrik övergång från lokalt till molnbaserade IT-system, Ur ett systemimplementatörsperspektiv är det individen och den individuella förmågan och kompetens. Ur ett projektgruppperspektiv är det att etablera en effektiv kommunikation.  Ur ett kundperspektiv är deras mognad och samsyn för projektet

Cloud computing has been given a great deal of attention during recent years. Almost all the technology market leaders and leading hosting service providers (like IBM, Microsoft and Verizon) have entered into the Cloud market as Cloud Providers. Cloud computing promises to provide highly available, secure, low cost, agile and highly scalable solution to the consumers. Scania is a global company and one of the world’s leading heavy vehicle manufacturers with 35,000+ employees. All the large organizations such as Scania, aim to constantly update themselves with the latest technology in order to meet their business requirements but, these organizations must always be convinced that there is a strong reason(s) to implement new technology. This research provides the method and criteria in relation to initiating Cloud computing. A number of Scania’s specific business requirements that it is possible to map to the Cloud are addressed in this thesis. The methodology of research is split in two parts. Firstly, the identification of business cases at Scania and their requirements with the Cloud and Secondly, the evaluation and comparison of the functionalities and capabilities of different vendors. The accumulated data is then compared and suitable vendors, according to those business requirements are suggested. This thesis also shares the experience of moving on premise applications to the Cloud. These are Scania specific applications which are currently being hosted in-house. The research also addresses the possibilities of portability between the Cloud providers. Although there is no standardization in relation to Cloud computing, some initiatives such as OpenStack are available and its current position and some application and data migration tools are also discussed. The thesis concludes with a general discussion, recommendations in relation to adapting Cloud computing and selecting the Cloud provider. This recommendation applies to every organization including Scania.

No description available.

Submitted by Bruna Rodrigues (bruna92rodrigues@yahoo.com.br) on 2016-10-21T11:00:36Z No. of bitstreams: 1 DissRSA.pdf: 2940714 bytes, checksum: 9af799d998ad9646a6f38b0d6e9c382a (MD5) / Approved for entry into archive by Marina Freitas (marinapf@ufscar.br) on 2016-11-08T18:44:27Z (GMT) No. of bitstreams: 1 DissRSA.pdf: 2940714 bytes, checksum: 9af799d998ad9646a6f38b0d6e9c382a (MD5) / Approved for entry into archive by Marina Freitas (marinapf@ufscar.br) on 2016-11-08T18:44:32Z (GMT) No. of bitstreams: 1 DissRSA.pdf: 2940714 bytes, checksum: 9af799d998ad9646a6f38b0d6e9c382a (MD5) / Made available in DSpace on 2016-11-08T18:44:39Z (GMT). No. of bitstreams: 1 DissRSA.pdf: 2940714 bytes, checksum: 9af799d998ad9646a6f38b0d6e9c382a (MD5) Previous issue date: 2016-08-10 / Não recebi financiamento / Cloud computing is a convenient computing model, because it allows the ubiquity with on-demand access to a set of configurable and shared features, that can be rapidly provisioned and made available with minimal effort or interaction with the service provider. IaaS is a different way to deliver cloud computing, where infrastructure servers, networking systems, storage, and all the necessary environment for the operating system to run the application are hired as services. Meanwhile, traditional companies still have doubts in relation to the transferring of their data outside of the limits of the corporation. The health of cloud computing systems is fundamental to the business, given the complexity of the systems it is difficult to ensure that all services and resources will work properly. In order to ensure a more appropriate management of the systems and services in the cloud, an architecture is proposed. The architecture has been modularized through specializing monitoring functions, data mining, and inference with Bayesian network. In this architecture are essential records of event monitoring systems and computing resources because the recorded data is mined to identify fault patterns a given system after the result of one or more events in the environment. For mining the monitoring data we proposed two algorithms, one for performing preprocessing of data and another to perform data transformation. As a data mining product obtained, data sets that were the input to create a Bayesian network. Through structural and parametric learning algorithms Bayesinas networks for each systems and services offered by cloud computing were created. The Bayesian network is intended to assist in decision making with prevention, prediction, error correction in systems and services, allowing to manage the health and performance of the most appropriate way systems. To check the compliance of the fault diagnosis of this architecture, we validate accuracy of inference of Bayesian network with cross-validation method using data sets generated by monitoring systems and services. / A computação em nuvem é um modelo de computação conveniente, pois permite a ubiquidade, com acesso sob demanda a um conjunto de recursos configuráveis e compartilhados, que podem ser rapidamente provisionados e disponibilizados com o mínimo de esforço ou interação com o fornecedor do serviço. IaaS é uma maneira diferente de entregar computação em nuvem, onde a infraestrutura de servidores, sistemas de rede, armazenamento e todo o ambiente necessário para o funcionamento do sistema operacional até aplicação são contratados como serviços. Entretanto, empresas tradicionais ainda possuem dúvidas com relação à transferência de seus dados para fora dos limites da corporação. A saúde de sistemas em computação em nuvem é algo fundamental para o negócio, e dada a complexidade dos sistemas é difícil garantir que todos os serviços e recursos funcionem adequadamente. A fim de garantir um gerenciamento mais adequado da saúde dos sistema e serviços na nuvem, propôs-se nesse trabalho uma arquitetura de diagnóstico de saúde de sistema de nuvem. A arquitetura foi modularizada, especializando funções de monitoramento, mineração de dados e inferência com rede Bayesiana. Nessa arquitetura, são fundamentais os registros de eventos de monitoramento dos sistemas e recursos computacionais, pois os dados registrados são minerados para identificar padrões de falhas. Para mineração dos dados de monitoramento foram propostos dois algoritmos: um para realizar a tarefa de pré- processamento dos dados e outro para realizar a transformação dos dados. Como produto da mineração dos dados, foram obtidos conjuntos de dados que foram o insumo para criar a rede Bayesiana. Por meio de algoritmos de aprendizagem estrutural e paramétrica foram criadas redes Bayesinas para cada sistema e disponibilizados por meio da computação em nuvem. A rede Bayesiana tem o objetivo de auxiliar na tomada de decis˜ao com prevenção, previsão, correção de falhas nos sistemas e serviços, permitindo assim gerenciar a saúde e o desempenho dos sistemas de forma mais adequada. Para verificar a aderência da arquitetura ao diagnóstico de falhas, validou-se a precisão de inferência da rede Bayesiana com o método de validação cruzada.

När en ny webbapplikation skall lanseras och driftsättas är det svårt att i förhand veta vilken datatrafik och belastning som tjänsten behöver vara dimensionerad för. Rapporten följer en webbapplikation som inte är förberedd för uppskalning till att bli separerad i olika komponenter för ökad skalbarhet och driftsäkerhet. I rapporten genomförs även en komparativ studie på olika typer av molntjänster som erbjuder infrastruktur (IaaS)-, plattform (PaaS)- och mjukvara (SaaS) som en tjänst. Målet med undersökningen var att hitta en kostnadseffektiv metod för att expandera applikationens infrastruktur och flytta implementationen till molnet. Resultatet och slutsatsen visar att den dyraste lösningen inte alltid är den bästa och i slutändan kan företag betalar pengar för resurser som de inte utnyttjar.

This thesis deals with the theme Moving information system in the cloud. The paper first describes the issue theoretically. It is defined and characterized by cloud computing as the current major trend in the implementation of information technology. A description of individual actors and models emerging in cloud computing, its brief history, related technology underlying or expand cloud computing, client issues and reference architecture model. It also includes an analysis of the current state offered cloud services, compared to open source and proprietary solutions, and was implemented insight into the upcoming National Strategy for Cloud Computing The case study move e-mail system to a hosted private cloud is demonstrated by the complexity of the entire process and the rugged environment of government institutions. On the basis of theoretical knowledge and case studies in the last chapter proposes a model that would in general help in deciding whether the institutions of government information systems migrate to cloud computing, which is also in line with the forthcoming national strategy.

This thesis discusses the new way of delivering IT services over the Internet widely known as cloud computing. In its opening part, cloud computing is put into a historical context of the evolution of enterprise computing, and the dominant issues the IT department faces today are mentioned. Further, the paper deals with several components that make up the architecture of cloud computing and reviews the benefits and drawbacks an enterprise can have while it adopts this new model. One of the primary aims of this thesis is to identify the impact of the technology trends on cloud computing. The thesis brings together four major computing trends, namely virtualization, multi-tenant architecture, service-oriented architecture and grid computing. Another aim is to focus on two trends related to IT infrastructure that will lead to fundamental changes in IT industry. The first of them is the emergence of extremely large-scale data centers at low cost locations, which can serve tremendous amount of customers and achieve considerable economies of scale. The second trend this paper points out is the shift from multi-purpose all-in-one computers into a wide range of mobile devices dedicated to a specific user's needs. The last aim of this thesis is to clarify the economic impact of cloud computing in terms of costs and changes in business models. The thesis concludes by evaluating the current adoption and predicting the future trend of cloud computing.

This thesis is focused on cloud technologies and their ever increasing implementation into corporate sector throughout industry branches, starting from cloud technology risks that are named and defined. This part is followed by business branches specification itself, their challenges, possibilities of solutions and risks that companies will probably encounter. Each chapter contains a summary of findings and proposal for acceptable solution.