Scientific Workflows (SWFs) are widely used to model applications in e-Science. In this programming model, scientific applications are described as a set of tasks that have dependencies among them. During the last decades, the execution of scientific workflows has been successfully performed in the available computing infrastructures (supercomputers, clusters and grids) using software programs called Workflow Management Systems (WMSs), which orchestrate the workload on top of these computing infrastructures. However, because each computing infrastructure has its own architecture and each scientific applications exploits efficiently one of these infrastructures, it is necessary to organize the way in which they are executed. WMSs need to get the most out of all the available computing and storage resources. Traditionally, scientific workflow applications have been extensively deployed in high-performance computing infrastructures (such as supercomputers and clusters) and grids. But, in the last years, the advent of cloud computing infrastructures has opened the door of using on-demand infrastructures to complement or even replace local infrastructures. However, new issues have arisen, such as the integration of hybrid resources or the compromise between infrastructure reutilization and elasticity, everything on the basis of cost-efficiency. The main contribution of this thesis is an ad-hoc solution for managing workflows exploiting the capabilities of cloud computing orchestrators to deploy resources on demand according to the workload and to combine heterogeneous cloud providers (such as on-premise clouds and public clouds) and traditional infrastructures (supercomputers and clusters) to minimize costs and response time. The thesis does not propose yet another WMS, but demonstrates the benefits of the integration of cloud orchestration when running complex workflows. The thesis shows several configuration experiments and multiple heterogeneous backends from a realistic comparative genomics workflow called Orthosearch, to migrate memory-intensive workload to public infrastructures while keeping other blocks of the experiment running locally. The running time and cost of the experiments is computed and best practices are suggested. / Los flujos de trabajo científicos son comúnmente usados para modelar aplicaciones en e-Ciencia. En este modelo de programación, las aplicaciones científicas se describen como un conjunto de tareas que tienen dependencias entre ellas. Durante las últimas décadas, la ejecución de flujos de trabajo científicos se ha llevado a cabo con éxito en las infraestructuras de computación disponibles (supercomputadores, clústers y grids) haciendo uso de programas software llamados Gestores de Flujos de Trabajos, los cuales distribuyen la carga de trabajo en estas infraestructuras de computación. Sin embargo, debido a que cada infraestructura de computación posee su propia arquitectura y cada aplicación científica explota eficientemente una de estas infraestructuras, es necesario organizar la manera en que se ejecutan. Los Gestores de Flujos de Trabajo necesitan aprovechar el máximo todos los recursos de computación y almacenamiento disponibles. Habitualmente, las aplicaciones científicas de flujos de trabajos han sido ejecutadas en recursos de computación de altas prestaciones (tales como supercomputadores y clústers) y grids. Sin embargo, en los últimos años, la aparición de las infraestructuras de computación en la nube ha posibilitado el uso de infraestructuras bajo demanda para complementar o incluso reemplazar infraestructuras locales. No obstante, este hecho plantea nuevas cuestiones, tales como la integración de recursos híbridos o el compromiso entre la reutilización de la infraestructura y la elasticidad, todo ello teniendo en cuenta que sea eficiente en el coste. La principal contribución de esta tesis es una solución ad-hoc para gestionar flujos de trabajos explotando las capacidades de los orquestadores de recursos de computación en la nube para desplegar recursos bajo demando según la carga de trabajo y combinar proveedores de computación en la nube heterogéneos (privados y públicos) e infraestructuras tradicionales (supercomputadores y clústers) para minimizar el coste y el tiempo de respuesta. La tesis no propone otro gestor de flujos de trabajo más, sino que demuestra los beneficios de la integración de la orquestación de la computación en la nube cuando se ejecutan flujos de trabajo complejos. La tesis muestra experimentos con diferentes configuraciones y múltiples plataformas heterogéneas, haciendo uso de un flujo de trabajo real de genómica comparativa llamado Orthosearch, para traspasar cargas de trabajo intensivas de memoria a infraestructuras públicas mientras se mantienen otros bloques del experimento ejecutándose localmente. El tiempo de respuesta y el coste de los experimentos son calculados, además de sugerir buenas prácticas. / Els fluxos de treball científics són comunament usats per a modelar aplicacions en e-Ciència. En aquest model de programació, les aplicacions científiques es descriuen com un conjunt de tasques que tenen dependències entre elles. Durant les últimes dècades, l'execució de fluxos de treball científics s'ha dut a terme amb èxit en les infraestructures de computació disponibles (supercomputadors, clústers i grids) fent ús de programari anomenat Gestors de Fluxos de Treballs, els quals distribueixen la càrrega de treball en aquestes infraestructures de computació. No obstant açò, a causa que cada infraestructura de computació posseeix la seua pròpia arquitectura i cada aplicació científica explota eficientment una d'aquestes infraestructures, és necessari organitzar la manera en què s'executen. Els Gestors de Fluxos de Treball necessiten aprofitar el màxim tots els recursos de computació i emmagatzematge disponibles. Habitualment, les aplicacions científiques de fluxos de treballs han sigut executades en recursos de computació d'altes prestacions (tals com supercomputadors i clústers) i grids. No obstant açò, en els últims anys, l'aparició de les infraestructures de computació en el núvol ha possibilitat l'ús d'infraestructures sota demanda per a complementar o fins i tot reemplaçar infraestructures locals. No obstant açò, aquest fet planteja noves qüestions, tals com la integració de recursos híbrids o el compromís entre la reutilització de la infraestructura i l'elasticitat, tot açò tenint en compte que siga eficient en el cost. La principal contribució d'aquesta tesi és una solució ad-hoc per a gestionar fluxos de treballs explotant les capacitats dels orquestadors de recursos de computació en el núvol per a desplegar recursos baix demande segons la càrrega de treball i combinar proveïdors de computació en el núvol heterogenis (privats i públics) i infraestructures tradicionals (supercomputadors i clústers) per a minimitzar el cost i el temps de resposta. La tesi no proposa un gestor de fluxos de treball més, sinó que demostra els beneficis de la integració de l'orquestració de la computació en el núvol quan s'executen fluxos de treball complexos. La tesi mostra experiments amb diferents configuracions i múltiples plataformes heterogènies, fent ús d'un flux de treball real de genòmica comparativa anomenat Orthosearch, per a traspassar càrregues de treball intensives de memòria a infraestructures públiques mentre es mantenen altres blocs de l'experiment executant-se localment. El temps de resposta i el cost dels experiments són calculats, a més de suggerir bones pràctiques. / Carrión Collado, AA. (2017). Management of generic and multi-platform workflows for exploiting heterogeneous environments on e-Science [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/86179 / TESIS

Multi-access Edge Computing (MEC) is a standard network architecture of edge computing, which is proposed to handle tremendous computation demands of emerging resource-intensive and latency-sensitive applications and services and accommodate Quality of Service (QoS) requirements for ever-growing users through computation offloading. Since the demand of end-users is unknown in a rapidly changing dynamic environment, processing offloaded tasks in a non-optimal server can deteriorate QoS due to high latency and increasing task failures. In order to deal with such a challenge in MEC, a two-stage Belief Rule-Based (BRB) workload orchestrator is proposed to distribute the workload of end-users to optimum computing units, support strict QoS requirements, ensure efficient utilization of computational resources, minimize task failures, and reduce the overall service time. The proposed BRB workload orchestrator decides the optimal execution location for each offloaded task from User Equipment (UE) within the overall MEC architecture based on network conditions, computational resources, and task requirements. EdgeCloudSim simulator is used to conduct comprehensive simulation experiments for evaluating the performance of the proposed BRB orchestrator in contrast to four workload orchestration approaches from the literature with different types of applications. Based on the simulation experiments, the proposed workload orchestrator outperforms state-of-the-art workload orchestration approaches and ensures efficient utilization of computational resources while minimizing task failures and reducing the overall service time.

Submitted by Nayara Passos (nayara.passos@ufpe.br) on 2015-03-10T14:26:26Z No. of bitstreams: 2 DISSERTAÇÃO Natália Cabral Silva.pdf: 2867606 bytes, checksum: 4e1c75788ce8db0420f34c1ca5195e63 (MD5) license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) / Made available in DSpace on 2015-03-11T17:30:06Z (GMT). No. of bitstreams: 2 DISSERTAÇÃO Natália Cabral Silva.pdf: 2867606 bytes, checksum: 4e1c75788ce8db0420f34c1ca5195e63 (MD5) license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) Previous issue date: 2014 / Diante do ambiente complexo e dinâmico encontrado nas empresas atualmente, o sistema tradicional de Workflow não está sendo flexível suficiente para modelar Processos de Negócio. Nesse contexto, surgiram os Processos Flexíveis que tem por principal objetivo suprir a necessidade de modelar processos menos estáticos. Processo declarativo é um tipo de processo flexível que permite os participantes decidirem a ordem em que as atividades são executadas através de regras de negócio. As regras de negócio determinam as restrições e obrigações que devem ser satisfeitas durante a execução. Tais regras descrevem o que deve ou não deve ser feito durante a execução do processo, mas não definem como. Os métodos e ferramentas atualmente disponíveis para modelar e executar processos declarativos apresentam várias limitações que prejudicam a sua utilização para este fim. Em particular, a abordagem que emprega lógica temporal linear (LTL) sofre do problema de explosão de estados a medida que o tamanho do modelo do processo cresce. Embora mecanismos eficientes em relação a memória terem surgido, eles não são capazes de adequadamente garantir a conclusão correta do processo, uma vez que permitem o usuário alcançar estados proibidos ou que causem deadlock. Além disso, as implementações atuais de ferramentas para execução de processos declarativos se concentram apenas em atividades manuais. Comunicação automática com aplicações externas para troca de dados e reutilização de funcionalidade não é suportado. Essas oportunidades de automação poderiam ser melhor exploradas por uma engine declarativa que se integra com tecnologias SOC existentes. Este trabalho propõe uma nova engine de regras baseada em grafo, chamado de REFlex. Tal engine não compartilha os problemas apresentados pelas abordagens disponíveis, sendo mais adequada para modelar processos de negócio declarativos. Além disso, REFlex preenche a lacuna entre os processos declarativos e SOC. O orquestrador REFlex é um orquestrador de serviços declarativo, eficiente e dependente de dados. Ele permite que os participantes chamem serviços externos para executar tarefas automatizadas. Diferente dos trabalhos relacionados, o algoritmo de REFlex não depende da geração de todos os estados alcançáveis, o que o torna adequado para modelar processos de negócios grandes e complexos. Além disso, REFlex suporta regras de negócio dependentes de dados, o que proporciona sensibilidade ao contexto. / Declarative business process modeling is a flexible approach to business process management in which participants can decide the order in which activities are performed. Business rules are employed to determine restrictions and obligations that must be satisfied during execution time. Such business rules describe what must or must not be done during the process execution, but do not prescribe how. In this way, complex control-flows are simplified and participants have more flexibility to handle unpredicted situations. The methods and tools currently available to model and execute declarative processes present several limitations that impair their use to this application. In particular, the well-known approach that employs Linear Temporal Logic (LTL) has the drawback of the state space explosion as the size of the process model grows. Although approaches proposing memory efficient methods have been proposed in the literature, they are not able to properly guarantee the correct termination of the process, since they allow the user to reach deadlock states. Moreover, current implementations of declarative business process engines focus only on manual activities. Automatic communication with external applications to exchange data and reuse functionality is barely supported. Such automation opportunities could be better exploited by a declarative engine that integrates with existing SOC technologies. This work proposes a novel graph-based rule engine called REFlex that does not share the problems presented by other engines, being better suited to model declarative business processes than the techniques currently in use. Additionally, such engine fills this gap between declarative processes and SOC. The REFlex orchestrator is an efficient, data-aware declarative web services orchestrator. It enables participants to call external web services to perform automated tasks. Different from related work, the REFlex algorithm does not depend on the generation of all reachable states, which makes it well suited to model large and complex business processes. Moreover, REFlex is capable of modeling data-dependent business rules, which provides unprecedented context awareness and modeling power to the declarative paradigm.

L’un des principaux défis des fournisseurs de services cloud est d’offrir aux utilisateurs une performance acceptable, tout en minimisant les besoins en matériel et énergie. Dans cette thèse CIFRE menée avec Outscale, un fournisseur de cloud, nous visons à optimiser l’allocation des ressources en utilisant de nouvelles sources d’information. Nous caractérisons la charge de travail pour comprendre le stress résultant sur l’orchestrateur, et la compétition pour les ressources disponibles qui dégrade la qualité de service. Nous proposons un modèle pour prédire la durée d’exécution des VMs à partir de caractéristiques prédictives disponibles au démarrage. Enfin, nous évaluons la sensibilité aux erreurs d’un algorithme de placement des VMs de la littérature qui se base sur ces prédictions. Nous ne trouvons pas d’intérêt à coupler note système prédictif avec cet algorithme, mais nous proposons d’autres façons d’utiliser les prédictions pour optimiser le placement des VMs. / One of the main challenges for cloud computing providers remains to offer trustable performance for all users, while maintaining an efficient use of hardware and energy resources. In the context of this CIFRE thesis lead with Outscale, apublic cloud provider, we perform an in-depth study aimed at making management algorithms use new sources of information. We characterize Outscale’s workload to understand the resulting stress for the orchestrator, and the contention for hardware resources. We propose models to predict the runtime of VMs based on features which are available when they start. We evaluate the sensitivity with respect to prediction error of a VM placement algorithm from the literature that requires such predictions. We do not find any advantage in coupling our prediction model and the selected algorithm, but we propose alternative ways to use predictions to optimize the placement of VMs.