Grade computacional é uma alternativa para melhorar o desempenho de aplicações paralelas, por permitir o uso simultâneo de vários recursos distribuídos. Entretanto, para que a utilização de uma grade seja adequada, é necessário que os recursos sejam utilizados de maneira a permitir a otimização de algum critério. Para isto, várias estratégias de escalonamento têm sido propostas, mas o grande desafio é extrair o potencial que os recursos oferecem para a execução de aplicações paralelas. Uma estratégia bastante usada em sistemas de escalonamento atuais é escalonar uma aplicação paralela nos recursos de um único cluster. Contudo, apesar da estratégia ser simples, ela é muito limitada, devido principalmente a baixa utilização dos recursos. Este trabalho propõe e implementa o sistema GCSE (Grid Cooperative Scheduling Environment) que provê uma estratégia de escalonamento cooperativo para usar eficientemente os recursos distribuídos. Os processos de uma aplicação paralela podem ser distribuídos em recursos de vários clusters e computadores, todos conectados a redes de comunicação públicas. GCSE também gerencia a execução das aplicações, bem como oferece um conjunto de primitivas que fornece informações sobre os ambientes de execução para o suporte à comunicação entre processos. Além disto, uma estratégia de antecipação de dados é proposta para aumentar ainda mais o desempenho das aplicações. Para realizar um bom escalonamento é preciso descobrir os recursos distribuídos. Neste sentido, o sistema LIMA (Light-weIght Monitoring Architecture) foi projetado e implementado. Este sistema provê um conjunto de estratégias e mecanismos para o armazenamento distribuído e acesso eficiente às informações sobre os recursos distribuídos. Além disto, LIMA adiciona facilidades de descobrimento e integração com o GCSE e outros sistemas. Por fim, serão apresentados os testes e avaliações dos resultados com o uso integrado dos sistemas GCSE e LIMA, compondo um ambiente robusto para a execução de aplicações paralelas. / Computing grid is an alternative for improving the parallel application performance, because it allows the simultaneous use of many distributed resources. However, in order to take advantage of a grid, the resources must be used in such a way that some criteria can be optimized. Thus, various scheduling strategies have been proposed, but the great challenge is the exploitation of the potential that the resources provide to the parallel application execution. A strategy often used in current scheduling systems is to schedule a parallel application on resources of a single cluster. Even though this strategy is simple, it is very limited, mainly due to low resource utilization. This thesis proposes and implements the GCSE system (Grid Cooperative Scheduling Environment) that provides a cooperative scheduling strategy for efficiently using the distributed resources. The processes of a parallel application can be distributed in resources of many clusters and computers, and they are all connected by public communication networks. GCSE also manages the application execution, as well as offering a primitive set that provide information about the execution environments for ensuring the communication between processes. Moreover, a data advancement strategy is proposed for improving the application performance. In order to perform a good scheduling, the distributed resources must be discovered. Therefore, the LIMA system (Light-weIght Monitoring Architecture) was designed and implemented. This system provides both strategy and mechanism set for distributed storage and efficient access to information about the distributed resources. In addition, LIMA offers facilities for resource discovering and integrating its functionalities both GCSE and other systems. Finally, the tests and result evaluations are presented with the integrated use of both GCSE and LIMA systems, composing a robust environment for executing parallel application.

Grade computacional é uma alternativa para melhorar o desempenho de aplicações paralelas, por permitir o uso simultâneo de vários recursos distribuídos. Entretanto, para que a utilização de uma grade seja adequada, é necessário que os recursos sejam utilizados de maneira a permitir a otimização de algum critério. Para isto, várias estratégias de escalonamento têm sido propostas, mas o grande desafio é extrair o potencial que os recursos oferecem para a execução de aplicações paralelas. Uma estratégia bastante usada em sistemas de escalonamento atuais é escalonar uma aplicação paralela nos recursos de um único cluster. Contudo, apesar da estratégia ser simples, ela é muito limitada, devido principalmente a baixa utilização dos recursos. Este trabalho propõe e implementa o sistema GCSE (Grid Cooperative Scheduling Environment) que provê uma estratégia de escalonamento cooperativo para usar eficientemente os recursos distribuídos. Os processos de uma aplicação paralela podem ser distribuídos em recursos de vários clusters e computadores, todos conectados a redes de comunicação públicas. GCSE também gerencia a execução das aplicações, bem como oferece um conjunto de primitivas que fornece informações sobre os ambientes de execução para o suporte à comunicação entre processos. Além disto, uma estratégia de antecipação de dados é proposta para aumentar ainda mais o desempenho das aplicações. Para realizar um bom escalonamento é preciso descobrir os recursos distribuídos. Neste sentido, o sistema LIMA (Light-weIght Monitoring Architecture) foi projetado e implementado. Este sistema provê um conjunto de estratégias e mecanismos para o armazenamento distribuído e acesso eficiente às informações sobre os recursos distribuídos. Além disto, LIMA adiciona facilidades de descobrimento e integração com o GCSE e outros sistemas. Por fim, serão apresentados os testes e avaliações dos resultados com o uso integrado dos sistemas GCSE e LIMA, compondo um ambiente robusto para a execução de aplicações paralelas. / Computing grid is an alternative for improving the parallel application performance, because it allows the simultaneous use of many distributed resources. However, in order to take advantage of a grid, the resources must be used in such a way that some criteria can be optimized. Thus, various scheduling strategies have been proposed, but the great challenge is the exploitation of the potential that the resources provide to the parallel application execution. A strategy often used in current scheduling systems is to schedule a parallel application on resources of a single cluster. Even though this strategy is simple, it is very limited, mainly due to low resource utilization. This thesis proposes and implements the GCSE system (Grid Cooperative Scheduling Environment) that provides a cooperative scheduling strategy for efficiently using the distributed resources. The processes of a parallel application can be distributed in resources of many clusters and computers, and they are all connected by public communication networks. GCSE also manages the application execution, as well as offering a primitive set that provide information about the execution environments for ensuring the communication between processes. Moreover, a data advancement strategy is proposed for improving the application performance. In order to perform a good scheduling, the distributed resources must be discovered. Therefore, the LIMA system (Light-weIght Monitoring Architecture) was designed and implemented. This system provides both strategy and mechanism set for distributed storage and efficient access to information about the distributed resources. In addition, LIMA offers facilities for resource discovering and integrating its functionalities both GCSE and other systems. Finally, the tests and result evaluations are presented with the integrated use of both GCSE and LIMA systems, composing a robust environment for executing parallel application.

Composição de serviços é um tópico que tem atraído cada vez mais o interesse por parte de pesquisadores na área de sistemas distribuídos. Além disso, o interesse por ambientes baseados em nuvem tem crescido significativamente conforme o seu uso aumenta e se firma como um importante modelo de negócios. Coreografias são formas de composições de serviços em que não há pontos centrais de falha; a responsabilidade da sua execução é distribuída entre os vários serviços componentes. Devido à natureza distribuída do fluxo de informações e dados de controle, o cumprimento de \\textit{Service Level Agreements} (SLAs) depende estritamente do monitoramento da Qualidade de Serviços (QoS), recursos virtuais da nuvem e mecanismos de reconfiguração dinâmica, capazes de automaticamente adaptar composições a mudanças de estado no sistema. Nesta dissertação, abordamos o estudo do gerenciamento de QoS em coreografias de serviços. Para isso desenvolvemos um sistema de middleware capaz de implantar e gerenciar o QoS de composições. Este teve seu desempenho avaliado utilizando o serviço Amazon EC2. Os resultados da avaliação mostram que com pouco esforço por parte dos desenvolvedores de composições, é possível cumprir o SLA de composições dentro do esperado utilizando escalabilidade horizontal ou vertical provida pelo middleware automaticamente. Adicionalmente, a nossa proposta traz economias em relação ao custo de implantação pois diminui a quantidade de recursos subutilizados. / Service composition has been a hot topic that has attracted the interesting of researchers in the distributed system area. Moreover, the interesting in cloud computing based environment has grown significantly. Its use has grown and it became to be a important business model. Choreographies are an specific kind of service composition that has no single point of failure; the responsibility of execution is distributed among the services. Due to the distributed nature of the these systems, the fulfilment of Service Level Agreements (SLAs) depends strictly on and automatic way to monitoring Quality of Service (QoS) and virtual computional resources as well as dinamic reconfiguration mechanisms, to be capable of automatically adapting compositions to changing environment. In this work, we studied QoS management in service choreographies. In order, we devised a middleware system capable of deploy service compositions and manage QoS of them. The middleware was evaluated using the Amazon EC2 cloud provider. The results shows that with less effort from the composition providers, it is possible to fulfil SLAs using horizontal or vertical scalability provided by the middleware automatically. Additionaly, our proposal brings up a cost reduction of deploy by decreasing the amount of underused resources.

Composição de serviços é um tópico que tem atraído cada vez mais o interesse por parte de pesquisadores na área de sistemas distribuídos. Além disso, o interesse por ambientes baseados em nuvem tem crescido significativamente conforme o seu uso aumenta e se firma como um importante modelo de negócios. Coreografias são formas de composições de serviços em que não há pontos centrais de falha; a responsabilidade da sua execução é distribuída entre os vários serviços componentes. Devido à natureza distribuída do fluxo de informações e dados de controle, o cumprimento de \\textit{Service Level Agreements} (SLAs) depende estritamente do monitoramento da Qualidade de Serviços (QoS), recursos virtuais da nuvem e mecanismos de reconfiguração dinâmica, capazes de automaticamente adaptar composições a mudanças de estado no sistema. Nesta dissertação, abordamos o estudo do gerenciamento de QoS em coreografias de serviços. Para isso desenvolvemos um sistema de middleware capaz de implantar e gerenciar o QoS de composições. Este teve seu desempenho avaliado utilizando o serviço Amazon EC2. Os resultados da avaliação mostram que com pouco esforço por parte dos desenvolvedores de composições, é possível cumprir o SLA de composições dentro do esperado utilizando escalabilidade horizontal ou vertical provida pelo middleware automaticamente. Adicionalmente, a nossa proposta traz economias em relação ao custo de implantação pois diminui a quantidade de recursos subutilizados. / Service composition has been a hot topic that has attracted the interesting of researchers in the distributed system area. Moreover, the interesting in cloud computing based environment has grown significantly. Its use has grown and it became to be a important business model. Choreographies are an specific kind of service composition that has no single point of failure; the responsibility of execution is distributed among the services. Due to the distributed nature of the these systems, the fulfilment of Service Level Agreements (SLAs) depends strictly on and automatic way to monitoring Quality of Service (QoS) and virtual computional resources as well as dinamic reconfiguration mechanisms, to be capable of automatically adapting compositions to changing environment. In this work, we studied QoS management in service choreographies. In order, we devised a middleware system capable of deploy service compositions and manage QoS of them. The middleware was evaluated using the Amazon EC2 cloud provider. The results shows that with less effort from the composition providers, it is possible to fulfil SLAs using horizontal or vertical scalability provided by the middleware automatically. Additionaly, our proposal brings up a cost reduction of deploy by decreasing the amount of underused resources.

Les techniques émergentes de l’ingénierie dirigée par les modèles et de la programmation générative ont permis la création de plusieurs générateurs (générateurs de code et compilateurs). Ceux-ci sont souvent utilisés afin de faciliter le développement logiciel et automatiser le processus de génération de code à partir des spécifications abstraites. De plus, les générateurs modernes comme les compilateurs C, sont devenus hautement configurables, offrant de nombreuses options de configuration à l'utilisateur de manière à personnaliser facilement le code généré pour la plateforme matérielle cible. Par conséquent, la qualité logicielle est devenue fortement corrélée aux paramètres de configuration ainsi qu'au générateur lui-même. Dans ce contexte, il est devenu indispensable de vérifier le bon comportement des générateurs. Cette thèse établit trois contributions principales : Contribution I: détection automatique des inconsistances dans les familles de générateurs de code : Dans cette contribution, nous abordons le problème de l'oracle dans le domaine du test non-fonctionnel des générateurs de code. La disponibilité de multiples générateurs de code avec des fonctionnalités comparables (c.-à-d. familles de générateurs de code) nous permet d'appliquer l'idée du test métamorphique en définissant des oracles de test de haut-niveau (c.-à-d. relation métamorphique) pour détecter des inconsistances. Une inconsistance est détectée lorsque le code généré présente un comportement inattendu par rapport à toutes les implémentations équivalentes de la même famille. Nous évaluons notre approche en analysant la performance de Haxe, un langage de programmation de haut niveau impliquant un ensemble de générateurs de code multi-plateformes. Les résultats expérimentaux montrent que notre approche est capable de détecter plusieurs inconsistances qui révèlent des problèmes réels dans cette famille de générateurs de code. Contribution II: une approche pour l'auto-configuration des compilateurs. Le grand nombre d'options de compilation des compilateurs nécessite une méthode efficace pour explorer l'espace d’optimisation. Ainsi, nous appliquons, dans cette contribution, une méta-heuristique appelée Novelty Search pour l'exploration de cet espace de recherche. Cette approche aide les utilisateurs à paramétrer automatiquement les compilateurs pour une architecture matérielle cible et pour une métrique non-fonctionnelle spécifique tel que la performance et l'utilisation des ressources. Nous évaluons l'efficacité de notre approche en vérifiant les optimisations fournies par le compilateur GCC. Nos résultats expérimentaux montrent que notre approche permet d'auto-configurer les compilateurs en fonction des besoins de l'utilisateur et de construire des optimisations qui surpassent les niveaux d'optimisation standard. Nous démontrons également que notre approche peut être utilisée pour construire automatiquement des niveaux d'optimisation qui représentent des compromis optimaux entre plusieurs propriétés non-fonctionnelles telles que le temps d'exécution et la consommation des ressources. Contribution III: Un environnement d'exécution léger pour le test et la surveillance de la consommation des ressources des logiciels. Enfin, nous proposons une infrastructure basée sur les micro-services pour assurer le déploiement et la surveillance de la consommation des ressources des différentes variantes du code généré. Cette contribution traite le problème de l'hétérogénéité des plateformes logicielles et matérielles. Nous décrivons une approche qui automatise le processus de génération, compilation, et exécution du code dans le but de faciliter le test et l'auto-configuration des générateurs. Cet environnement isolé repose sur des conteneurs système, comme plateformes d'exécution, pour une surveillance et analyse fine des propriétés liées à l'utilisation des ressources (CPU et mémoire). / Generative software development has paved the way for the creation of multiple generators (code generators and compilers) that serve as a basis for automatically producing code to a broad range of software and hardware platforms. With full automatic code generation, users are able to rapidly synthesize software artifacts for various software platforms. In addition, they can easily customize the generated code for the target hardware platform since modern generators (i.e., C compilers) become highly configurable, offering numerous configuration options that the user can apply. Consequently, the quality of generated software becomes highly correlated to the configuration settings as well as to the generator itself. In this context, it is crucial to verify the correct behavior of generators. Numerous approaches have been proposed to verify the functional outcome of generated code but few of them evaluate the non-functional properties of automatically generated code, namely the performance and resource usage properties. This thesis addresses three problems : (1) Non-functional testing of generators: We benefit from the existence of multiple code generators with comparable functionality (i.e., code generator families) to automatically test the generated code. We leverage the metamorphic testing approach to detect non-functional inconsistencies in code generator families by defining metamorphic relations as test oracles. We define the metamorphic relation as a comparison between the variations of performance and resource usage of code, generated from the same code generator family. We evaluate our approach by analyzing the performance of HAXE, a popular code generator family. Experimental results show that our approach is able to automatically detect several inconsistencies that reveal real issues in this family of code generators. (2) Generators auto-tuning: We exploit the recent advances in search-based software engineering in order to provide an effective approach to tune generators (i.e., through optimizations) according to user's non-functional requirements (i.e., performance and resource usage). We also demonstrate that our approach can be used to automatically construct optimization levels that represent optimal trade-offs between multiple non-functional properties such as execution time and resource usage requirements. We evaluate our approach by verifying the optimizations performed by the GCC compiler. Our experimental results show that our approach is able to auto-tune compilers and construct optimizations that yield to better performance results than standard optimization levels. (3) Handling the diversity of software and hardware platforms in software testing: Running tests and evaluating the resource usage in heterogeneous environments is tedious. To handle this problem, we benefit from the recent advances in lightweight system virtualization, in particular container-based virtualization, in order to offer effective support for automatically deploying, executing, and monitoring code in heterogeneous environment, and collect non-functional metrics (e.g., memory and CPU consumptions). This testing infrastructure serves as a basis for evaluating the experiments conducted in the two first contributions.

Modern science relies on workflow technology to capture, process, and analyse data obtained from scientific instruments. Scientific workflows are precise descriptions of experiments in which multiple computational tasks are coordinated based on the dataflows between them. Orchestrating scientific workflows presents a significant research challenge: they are typically executed in a manner such that all data pass through a centralised computer server known as the engine, which causes unnecessary network traffic that leads to a performance bottleneck. These workflows are commonly composed of services that perform computation over geographically distributed resources, and involve the management of dataflows between them. Centralised orchestration is clearly not a scalable approach for coordinating services dispersed across distant geographical locations. This thesis presents a scalable decentralised service-oriented orchestration system that relies on a high-level data coordination language for the specification and execution of workflows. This system's architecture consists of distributed engines, each of which is responsible for executing part of the overall workflow. It exploits parallelism in the workflow by decomposing it into smaller sub-workflows, and determines the most appropriate engines to execute them using computation placement analysis. This permits the workflow logic to be distributed closer to the services providing the data for execution, which reduces the overall data transfer in the workflow and improves its execution time. This thesis provides an evaluation of the presented system which concludes that decentralised orchestration provides scalability benefits over centralised orchestration, and improves the overall performance of executing a service-oriented workflow.

The notions of "network embedded cloud", also known as a "network enabled cloud" or a "carrier cloud", is an emerging technology trend aiming to integrate network services while exploiting the on-demand nature of the cloud paradigm. A network embedded cloud is a distributed cloud environment where data centers are distributed at the edge of the operator's network. Distributing data centers or computing resources across the network introduces topological and geographical locality dependency. In the case of a network enabled cloud, in addition to the information regarding available processing, memory, and storage capacity, resource management requires information regarding the network's topology and available bandwidth on the links connecting the different nodes of the distributed cloud. This thesis project designed, implemented, and evaluated the use of open shortest path first with traffic engineering (OSPF-TE) for propagating the resource status in a network enabled cloud. The information carried over OSPF-TE are used for network-aware scheduling of virtual machines. In particular, OSPF-TE was extended to convey virtualization and processing related information to all the nodes in the network enabled cloud. Modeling, emulation, and analysis shows the proposed solution can provide the required data to a cloud management system by sending a data center's resources information in the form of new opaque link-state advertisement with a minimum interval of 5 seconds. In this case, each embedded data centers injects a maximum 38.4 bytes per second of additional traffic in to the network.<p> / Ett "network embedded cloud", även känt som ett "network enabled cloud" eller ett "carrier cloud", är en ny teknik trend som syftar till att tillhandahålla nätverkstjänster medan on-demand egenskapen av moln-paradigmet utnyttjas.  Traditionella telekommunikationsapplikationer bygger ofta på en distributed service model och kan använda ett "network enabled cloud" som dess exekverande plattform. Dock kommer sådana inbäddade servrar av naturliga skäl vara geografiskt utspridda, varför de är beroende av topologisk och geografisk lokalisering. Detta ändrar på resurshanteringsproblemet jämfört med resurshantering i datacentrum. I de fall med ett network enabled cloud, utöver informationen om tillgängliga CPU, RAM och lagring, behöver resursfördelningsfunktionen information om nätverkets topologi och tillgänglig bandbredd på länkarna som förbinder de olika noderna i det distribuerade molnet. Detta examensarbete har utformat, tillämpat och utvärderat ett experiment-orienterad undersökning av användningen av open shortest path first med traffich engineering (OSPF-TE) för resurshantering i det network enabled cloud. I synnerhet utvidgades OSPF-TE till att förmedla virtualisering och behandla relaterad information till alla noder i nätverket. Detta examensarbete utvärderar genomförbarheten och lämpligheten av denna metod, dess flexibilitet och prestanda. Analysen visade att den föreslagna lösningen kan förse nödvändiga uppgifter till cloud management system genom att skicka ett datacenters resursinformation i form av ny opaque LSA (kallat Cloud LSA) med ett minimumintervall av 5 sekunder och maximal nätverksbelastning av 38,4 byte per sekund per inbäddade data center.