Weather forecasting models are computationally intensive applications and traditionally they are executed in parallel machines. However, some issues prevent these models from fully exploiting the available computing power. One of such issues is load imbalance, i.e., the uneven distribution of load across the processors of the parallel machine. Since weather models are typically synchronous applications, that is, all tasks synchronize at every time-step, the execution time is determined by the slowest task. The causes of such imbalance are either static (e.g. topography) or dynamic (e.g. shortwave radiation, moving thunderstorms). Various techniques, often embedded in the application’s source code, have been used to address both sources. However, these techniques are inflexible and hard to use in legacy codes. In this thesis, we explore the concept of processor virtualization for dynamically balancing the load in weather models. This means that the domain is over-decomposed in more tasks than the available processors. Assuming that many tasks can be safely executed in a single processor, each processor is put in charge of a set of tasks. In addition, the system can migrate some of them from overloaded processors to underloaded ones when it detects load imbalance. This approach has the advantage of decoupling the application from the load balancing strategy. Our objective is to show that processor virtualization can be applied to weather models as long as an appropriate strategy for migrations is used. Our proposal takes into account the communication pattern of the application in addition to the load of each processor. In this text, we present the techniques used to minimize the amount of change needed in order to apply processor virtualization to a real-world application. Furthermore, we analyze the effects caused by the frequency at which the load balancer is invoked and a threshold that activates rebalancing. We propose an automatic strategy to find an optimal threshold to trigger load balancing. These strategies are centralized and work well for moderately large machines. For larger machines, we present a fully distributed algorithm and analyze its performance. As a study case, we demonstrate the effectiveness of our approach for dynamically balancing the load in Brams, a mesoscale weather forecasting model based on MPI parallelization. We choose this model because it presents a considerable load imbalance caused by localized thunderstorms. In addition, we analyze how other effects of processor virtualization can improve performance.

La conception d'applications scientifiques à base de couplage de code est une tâche complexe car elle demande de concilier une facilité de programmation et une obtention de haute performance. En outre, les ressources matérielles (supercalculateurs, grappes de calcul, grilles) permettant leur exécution forment un ensemble hétérogène en constante évolution. Les modèles à base de composants logiciels forment une piste prometteuse pour gérer ces deux sources de complexité car ils permettent d’exprimer les interactions entre les différents constituants d’une application tout en offrant des possibilités d'abstraction des ressources. Néanmoins, les modèles existants ne permettent pas d'exprimer de manière satisfaisante les applications constituées de motifs répliqués dynamiques et hiérarchiques. Ainsi, cette thèse vise à améliorer l'existant – et en particulier la plate-forme générique de simulation numérique SALOME – pour une classe d'applications très répandue : les applications à base de décomposition de domaine et la variante utilisant le raffinement de maillage adaptatif. Tout d'abord, nous avons proposé d’étendre le modèle de composition spatial et temporel de SALOME en ajoutant la possibilité de définir dynamiquement la cardinalité des composants. Cela demande en particulier de gérer les communications de groupes ainsi induites. La proposition a été implémentée dans SALOME et validée via une application de décomposition de domaine à base de couplage de plusieurs instances de Code_Aster. Ensuite, nous avons étudié la pertinence d'utiliser un modèle de composant supportant des connecteurs natifs (MPI, mémoire partagée, appel de méthode) pour permettre une composition plus fine des interactions entre composants. Les résultats d'expériences montrent que des performances équivalentes aux versions natives sont obtenues tout en permettant de manipuler facilement l'architecture de l'application. Enfin, nous avons étudié les extensions nécessaires aux modèles à composants (abstraction,hiérarchie, dynamicité) pour la conception d’applications de raffinement de maillage adaptatif. Les modèles de composants spatio-temporels les plus avancés permettent ainsi d'exprimer ce type d'application mais les performances sont limitées par leur mise en œuvre centralisée ainsi que par le manque de moyens efficaces pour modifier à la volée des assemblages de composants. / Designing scientific applications based on code coupling is a complex task. It requires both an easy programming process and high-performance. In addition, execution resources (supercomputers, computer clusters, grids)  are heterogeneous and constantly evolving. Software components models offer a promising perspective to manage this double complexity because they can express interactions between the different parts of an application while providing abstraction of resources. However, existing models cannot accurately express the applications made of dynamic and hierarchical patterns. The aim of this thesis is to improve the existing models, and in particular the generic platform for numerical simulation SALOME, for a class of widespread applications : applications based on domain decomposition, and its dynamic variant using adaptive mesh refinement. Firstly, we proposed to extend the spatial and temporal composition model provided by SALOME, by adding the ability to dynamically set component cardinality. This requires in particular to manage group communications induced. The proposal has been implemented into SALOME and validated via a domain decomposition application based on coupling several instances of Code_Aster.Then, we have studied the relevance of using a component model supporting native connectors (MPI, shared memory, method invocation), in order to allow finer composition interactions between components.The experiment results show that performances obtained are equivalent to those of the native versions, while allowing to easily manipulate the application architecture. Finally, we studied the necessary component models extensions (abstraction, hierarchy, dynamicity) for designing adaptative mesh refinement applications. The most advanced spatio-temporal component models can express this type of application but performances are limited by their centralized implementation and by the lack of efficient ways of modifying component assembling at execution time.

Les ressources de calcul à haute performance (High-Performance Computing—HPC), telles que les supercalculateurs, les grappes, les grilles de calcul ou les Clouds HPC, sont gérées par des gestionnaires de ressources (Resource Management System—RMS) qui multiplexent les ressources entre plusieurs utilisateurs et décident comment allouer les nœuds de calcul aux applications des utilisateurs. Avec la multiplication de machines péta-flopiques et l’arrivée des machines exa-flopiques attendue en 2020, l’optimisation de l’allocation des ressources aux applications est essentielle pour assurer que leur exécution soit efficace. Cependant, les RMSs existants, tels que les batch schedulers, n’offrent qu’une interface restreinte. Dans la plupart des cas, l’application doit choisir les ressources « aveuglément » lors de la soumission sans pouvoir adapter son choix à l’état des ressources ciblées, ni avant, ni pendant l’exécution.Le but de cette Thèse est d’améliorer la gestion des ressources, afin de permettre aux applications d’allouer des ressources efficacement. Pour y parvenir, nous proposons des architectures logicielles qui favorisent la collaboration entre les applications et le gestionnaire de ressources, permettant ainsi aux applications de négocier les ressources qu’elles veulent utiliser. À cette fin, nous analysons d’abord les types d’applications et leurs besoins en ressources, et nous les divisons en plusieurs catégories : rigide, modelable, malléable et évolutive. Pour chaque cas, nous soulignons les opportunités d’amélioration de la gestion de ressources. Une première contribution traite les applications modelables, qui négocient les ressources seulement avant leur démarrage. Nous proposons CooRMv1, une architecture RMS centralisée, qui délègue la sélection des ressources aux lanceurs d’application. Des simulations montrent qu’un tel système se comporte bien en termes d’extensibilité et d’équité. Les résultats ont été validés avec un prototype déployé sur la plate-forme Grid’5000. Une deuxième contribution se focalise sur la négociation des allocations pour des ressources géographiquement distribuées qui appartiennent à plusieurs institutions. Nous étendons CooRMv1 pour proposer distCooRM, une architecture RMS distribuée, qui permet aux applications modelables de co-allouer efficacement des ressources gérées par plusieurs agents indépendants. Les résultats de simulation montrent que distCooRM se comporte bien et passe à l’échelle pour un nombre raisonnable d’applications. Ensuite, nous nous concentrons sur la négociation des ressources à l’exécution pour mieux gérer les applications malléables et évolutives. Nous proposons CooRMv2, une architecture RMS centralisée, qui permet l’ordonnancement efficace des applications évolutives, et surtout celles dont l’évolution n’est pas prévisible. Une application peut faire des « pré-allocations » pour exprimer ses pics de besoins en ressources. Cela lui permet de demander dynamiquement des ressources, dont l’allocation est garantie tant que la pré-allocation n’est pas dépassée. Les ressources pré-allouées mais inutilisées sont à la disposition des autres applications. Des gains importants sont ainsi obtenus, comme les simulations que nous avons effectuées le montrent.Enfin, nous partons de logiciels utilisés en production pour illustrer l’intérêt, mais aussi la difficulté, d’améliorer la collaboration entre deux systèmes existants. Avec GridTLSE comme application et DIET comme RMS, nous avons trouvé un cas d’utilisation mal supporté auparavant. Nous identifions le problème sous-jacent d’ordonnancement des calculs optionnels et nous proposons une architecture pour le résoudre. Des expériences réelles sur la plate-forme Grid’5000 montrent que plusieurs métriques peuvent être améliorées, comme par exemple la satisfaction des utilisateurs, l’équité et le nombre de requêtes traitées. En outre, nous montrons que cette solution présente une bonne extensibilité. / High-Performance Computing (HPC) resources, such as Supercomputers, Clusters, Grids and HPC Clouds, are managed by Resource Management Systems (RMSs) that multiple resources among multiple users and decide how computing nodes are allocated to user applications. As more and more petascale computing resources are built and exascale is to be achieved by 2020, optimizing resource allocation to applications is critical to ensure their efficient execution. However, current RMSs, such as batch schedulers, only offer a limited interface. In most cases, the application has to blindly choose resources at submittal without being able to adapt its choice to the state of the target resources, neither before it started nor during execution. The goal of this Thesis is to improve resource management, so as to allow applications to efficiently allocate resources. We achieve this by proposing software architectures that promote collaboration between the applications and the RMS, thus, allowing applications to negotiate the resources they run on. To this end, we start by analysing the various types of applications and their unique resource requirements, categorizing them into rigid, moldable, malleable and evolving. For each case, we highlight the opportunities they open up for improving resource management.The first contribution deals with moldable applications, for which resources are only negotiated before they start. We propose CooRMv1, a centralized RMS architecture, which delegates resource selection to the application launchers. Simulations show that the solution is both scalable and fair. The results are validated through a prototype implementation deployed on Grid’5000. Second, we focus on negotiating allocations on geographically-distributed resources, managed by multiple institutions. We build upon CooRMv1 and propose distCooRM, a distributed RMS architecture, which allows moldable applications to efficiently co-allocate resources managed by multiple independent agents. Simulation results show that distCooRM is well-behaved and scales well for a reasonable number of applications. Next, attention is shifted to run-time negotiation of resources, so as to improve support for malleable and evolving applications. We propose CooRMv2, a centralized RMS architecture, that enables efficient scheduling of evolving applications, especially non-predictable ones. It allows applications to inform the RMS about their maximum expected resource usage, through pre-allocations. Resources which are pre-allocated but unused can be filled by malleable applications. Simulation results show that considerable gains can be achieved. Last, production-ready software are used as a starting point, to illustrate the interest as well as the difficulty of improving cooperation between existing systems. GridTLSE is used as an application and DIET as an RMS to study a previously unsupported use-case. We identify the underlying problem of scheduling optional computations and propose an architecture to solve it. Real-life experiments done on the Grid’5000 platform show that several metrics are improved, such as user satisfaction, fairness and the number of completed requests. Moreover, it is shown that the solution is scalable.

Weather forecasting models are computationally intensive applications and traditionally they are executed in parallel machines. However, some issues prevent these models from fully exploiting the available computing power. One of such issues is load imbalance, i.e., the uneven distribution of load across the processors of the parallel machine. Since weather models are typically synchronous applications, that is, all tasks synchronize at every time-step, the execution time is determined by the slowest task. The causes of such imbalance are either static (e.g. topography) or dynamic (e.g. shortwave radiation, moving thunderstorms). Various techniques, often embedded in the application’s source code, have been used to address both sources. However, these techniques are inflexible and hard to use in legacy codes. In this thesis, we explore the concept of processor virtualization for dynamically balancing the load in weather models. This means that the domain is over-decomposed in more tasks than the available processors. Assuming that many tasks can be safely executed in a single processor, each processor is put in charge of a set of tasks. In addition, the system can migrate some of them from overloaded processors to underloaded ones when it detects load imbalance. This approach has the advantage of decoupling the application from the load balancing strategy. Our objective is to show that processor virtualization can be applied to weather models as long as an appropriate strategy for migrations is used. Our proposal takes into account the communication pattern of the application in addition to the load of each processor. In this text, we present the techniques used to minimize the amount of change needed in order to apply processor virtualization to a real-world application. Furthermore, we analyze the effects caused by the frequency at which the load balancer is invoked and a threshold that activates rebalancing. We propose an automatic strategy to find an optimal threshold to trigger load balancing. These strategies are centralized and work well for moderately large machines. For larger machines, we present a fully distributed algorithm and analyze its performance. As a study case, we demonstrate the effectiveness of our approach for dynamically balancing the load in Brams, a mesoscale weather forecasting model based on MPI parallelization. We choose this model because it presents a considerable load imbalance caused by localized thunderstorms. In addition, we analyze how other effects of processor virtualization can improve performance.

O uso de clusters e grids tem beneficiado durante anos a comunidade de computação de alto desempenho (HPC). O uso deste tipo de sistemas tem permitido aos cientistas usar conjuntos de dados maiores para executar cálculos mais complexos. A computação de alto desempenho tem ajudado para obter aqueles resultados em menos tempo, mas aumentou o custo das despesas de capital nesta área da ciência. Como alguns projetos de e-science são realizados também em ambientes de rede altamente distribuídos, ou usando conjuntos de dados imensos que muitas vezes requerem computação em grade, eles são muito bons candidatos para as iniciativas de computação em nuvem. O paradigma Cloud Computing surgiu como uma solução prática com foco comercial para realizar computação científica em larga escala. A elasticidade da nuvem e o modelo pay-as-you-go apresenta uma oportunidade interessante para aplicações comumente executados em supercomputadores ou clusters. Esta tese apresenta e avalia os desafios da migração e execução da previsão numérica de tempo (NWP) numa infra-estrutura de computação em nuvem. Foi realizada a migração desta aplicação HPC e foi avaliado o desempenho em um cluster local e na nuvem utilizando diferentes tamanhos de instâncias virtuais. Analisamos as principais características da aplicação executando na nuvem. As experiências demonstram que, embora o processamento e a rede criam um fator limitante, o armazenamento dos conjuntos de dados de entrada e saída na nuvem apresentam uma opção atraente para compartilhar resultados e facilitar a implantação de um ambiente de ensaio para investigação meteorológica. Os resultados mostram que a infraestrutura de nuvem pode ser usada como uma alternativa viável de HPC para software de previsão numérica do tempo. / The usage of clusters and grids has benefited for years the High Performance Computing (HPC) community. These kind of systems have allowed scientists to use bigger datasets and to perform more intensive computations, helping them to achieve results in less time but has also increased the upfront costs associated with this area of science. As some e-Science projects are carried out also in highly distributed network environments or using immense data sets that sometimes require grid computing, they are good candidates for cloud computing initiatives. The Cloud Computing paradigm has emerged as a practical solution to perform large-scale scientific computing. The elasticity of the cloud and its pay-as-you-go model presents an attractive opportunity for applications commonly executed in clusters or supercomputers. In this context, the user does not need to buy infrastructure, the resources can be rented from a provider and used for a period of time. This thesis presents the challenges and solutions of migrating a numerical weather prediction (NWP) application to a cloud computing infrastructure. We performed the migration of this HPC application and evaluated its performance in a local cluster and the cloud using different instance sizes. We analyzed the main characteristics of the application running in the cloud. The experiments demonstrate that, although processing and networking create a limiting factor, storing input and output datasets in the cloud presents an attractive option to share results and ease the deployment of a test-bed for a weather research platform. Results show that cloud infrastructure can be used as a viable HPC alternative for numerical weather prediction software.

Computação em nuvem é um novo paradigma, onde recursos computacionais são disponibilizados como serviços. Neste cenário, o usuário não tem a necessidade de adquirir infraestrutura, ele pode alugar os recursos de um provedor e usá-los durante um certo período de tempo. Além disso, o usuário pode facilmente alocar e desalocar quantos recursos ele desejar, num ambiente totalmente elástico. O usuário só é cobrado pelo efetivo uso que for feito dos recursos alocados, isso significa que ele somente pagará pelo que for utilizado. Por outro lado, usuários de processamento de alto desempenho (PAD) tem a necessidade de utilizar grande poder computacional como uma ferramenta de trabalho. Para se ter acesso a estes recursos, são necessários investimentos financeiros adequados para aquisição de sistemas para PAD. Mas, neste caso, duas situações podem incorrer em problemas. O usuário necessita ter acesso aos recursos financeiros totais para adquirir e manter um sistema para PAD, e esses recusros são limitados. O propósito dessa dissertação é avaliar se o paradigma de computação em nuvem é um ambiente viável para PAD, verificando se este modelo de computação tem a capaciodade de prover acesso a ambientes que podem ser utilizados para a execução de aplicações de alto desempenho, e também, se o custo benefício apresentado é melhor do que o de sistemas tradicionais. Para isso, todo o modelo de computação em nuvem foi avaliado para se identificar quais partes dele tem o potencial para ser usado para PAD. Os componentes identificados foram avaliados utilizando-se proeminentes provedores de computação em nuvem. Foram analisadas as capacidades de criação de ambientes de PAD, e tais ambientes tiveram seu desempenho analisado através da utilização de técnicas tradicionais. Para a avaliação do custo benefício, foi criado e aplicado um modelo de custo. Os resultados mostraram que todos os provedores analisados possuem a capacidade de criação de ambientes de PAD. Em termos de desempenho, houveram alguns casos em que os provedores de computação em nuvem foram melhores do que um sistema tradicional. Na perspectiva de custo, a nuvem apresenta uma alternativa bastante interessante devido ao seu modelo de cobrança de acordo com o uso. Como conclusão dessa dissertação, foi mostrado que a computação em nuvem pode ser utilizada como uma alternativa real para ambientes de PAD. / Cloud computing is a new paradigm, where computational resources are offered as services. In this context, the user does not need to buy infrastructure, the resources can be rented from a provider and used for a period of time. Furthermore the user can easily allocate as many resources as needed, and deallocate them as well, in a totally elastic environment. The resources need to be paid only for the effective usage time. On the other hand, High-Performance Computing (HPC) requires a large amount of computational power. To acquire systems capable for HPC, large financial investments are necessary. Apart from the initial investment, the user must pay the maintenance costs, and has only limited computational resources. To overcome these issues, this thesis aims to evaluate the cloud computing paradigm as a candidate environment for HPC. We analyze the efforts and challenges for porting and deploy HPC applications to the cloud. We evaluate if this computing model can provide sufficient capacities for running HPC applications, and compare its cost efficiency to traditional HPC systems, such as clusters. The cloud computing paradigm was analyzed to identify which models have the potential to be used for HPC purposes. The identified models were then evaluated using major cloud providers, Microsoft Windows Azure, Amazon EC2 and Rackspace and compare them to a traditional HPC system. We analyzed the capabilities to create HPC environments, and evaluated their performance. For the evaluation of the cost efficiency, we developed an economic model. The results show that all the evaluated providers have the capability to create HPC environments. In terms of performance, there are some cases where cloud providers present a better performance than the traditional system. From the cost perspective, the cloud presents an interesting alternative due to the pay-per-use model. Summarizing the results, this dissertation shows that cloud computing can be used as a realistic alternative for HPC environments.

Submitted by Fabio Sobreira Campos da Costa (fabio.sobreira@ufpe.br) on 2016-07-28T12:46:53Z No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) tese_diogotc_final.pdf: 7303783 bytes, checksum: f9959e8bb63b91d247de9903c2484d35 (MD5) / Made available in DSpace on 2016-07-28T12:46:53Z (GMT). No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) tese_diogotc_final.pdf: 7303783 bytes, checksum: f9959e8bb63b91d247de9903c2484d35 (MD5) Previous issue date: 2016-01-29 / Esta tese apresenta uma metodologia de paralelização híbrida aplicada ao Método dos Elementos Discretos (DEM - Discrete Element Method) que combina MPI e OpenMP com o intuito de melhoria de desempenho computacional. A metodologia utiliza estratégias de decomposição de domínio visando a distribuição do cálculo de modelos de larga escala em um cluster. A técnica proposta também particiona a carga de trabalho de cada subdomínio entre threads. Este procedimento adicional visa obter maiores desempenhos computacionais através do ajuste de utilização de mecanismos de troca de mensagens entre processos e paralelização por threads. O objetivo principal da técnica é reduzir os elevados tempos de comunicação entre processos em ambientes computacionais de memória compartilhada tais como os processadores modernos. A divisão de trabalho por threads emprega a curva de preenchimento de espaço de Hilbert (HSFC) visando a melhoria de localidade dos dados e evitando custos computacionais (overheads) resultantes de ordenações constantes para o vetor de partículas. As simulações numéricas apresentadas permitem avaliar os métodos de decomposição de domínio, técnicas de particionamento, mecanismos de controle de acesso à memória, dentre outros. Algoritmos distintos de particionamento e diferentes estratégias de solução paralela são abordados para ambientes computacionais de memória distribuída, compartilhada ou para um modelo híbrido que envolve os dois ambientes. A metodologia desenvolvida e a ferramenta computacional utilizada nas implementações realizadas, o software DEMOOP, fornecem recursos que podem ser aplicados em diversos problemas de engenharia envolvendo modelos de partículas em larga escala. Nesta tese alguns destes problemas são abordados, em especial aqueles relacionados com fluxo de partículas em rampas, em funis de descarga e em cenários reais de deslizamento de terra. Os resultados mostram que as estratégias de execução híbridas atingem, em geral, melhores desempenhos computacionais que aqueles que se baseiam unicamente em troca de mensagens. A técnica de paralelização híbrida desenvolvida também obtém um bom controle de balanço de carga entre threads. Os estudos de caso apresentados apresentam boa escalabilidade e eficiências paralelas. O método proposto permite uma execução configurável de modelos numéricos do DEM e introduz uma estratégia combinada que melhora localidade dos dados e um balanceamento de carga iterativo. / This thesis introduces a methodology of hybrid parallelization applied to the Discrete Element Method (DEM) that combines MPI and OpenMP to improve computational performance. The methodology uses domain decomposition strategies to distribute the computation of large-scale models in a cluster. It also partitions the workload of each subdomain among threads. This additional procedure aims to reach higher computational performance by adjusting the usage of message passing artifacts and threads. The main objective is to reduce the expensive communications between processes in computer resources of shared memory such as modern processors. The work division by threads employs Hilbert Space Filling Curves (HSFC) in order to improve data-locality and to avoid the overhead caused by the dynamical sorting of the particles array. Presented numerical simulations allow to evaluate several domain decomposition schemes, partitioning methods, mechanisms of memory access control, among others. The work investigate distinct schemes of parallel solution for both distributed and shared memory environments. The method and the computational tool employed, the software DEMOOP, provide applied resources for several engineering problems involving large scale particle models. Some of these problems are presented on this thesis, such as the particle flows that happen on inclined ramps, discharge hoppers and real scenarios of landslides. The results shows that the hybrid executions reach better computational performance than those based on message passing only, including a good control of load balancing among threads. Case studies present good scalability and parallel efficiencies. The proposed approach allows a configurable execution of numerical models and introduces a combined scheme that improves data-locality and an iterative workload balancing.

FundaÃÃo Cearense de Apoio ao Desenvolvimento Cientifico e TecnolÃgico / A contribuiÃÃo das tradicionais bibliotecas cientÃcas mostra-se consolidada na construÃÃo de aplicaÃÃes de alto desempenho. No entanto, tal artefato de desenvolvimento possui algumas limitaÃÃes de integraÃÃo, de produtividade em aplicaÃÃes de larga escala e de exibilidade para mudanÃas no contexto do problema. Por outro lado, a tecnologia de desenvolvimento baseada em componentes, recentemente proposta como alternativa viÃvel para a arquitetura de aplicaÃÃes de ComputaÃÃo de Alto Desempenho (CAD), tem fornecido meios para superar esses desaos. Vemos assim, que as bibliotecas cientÃcas e a programaÃÃo orientada a componentes sÃo tÃcnicas complementares na melhoria do processo de desenvolvimento de aplicaÃÃes modernas de CAD. Dessa forma, este trabalho tem por objetivo propor um mÃtodo sistemÃtico para integraÃÃo de bibliotecas cientÃcas sobre a plataforma de componentes paralelos HPE (Hash Programming Environment ), buscando oferecer os aspectos vantajosos complementares do uso de componentes e de bibliotecas cientÃcas aos desenvolvedores de programas paralelos que implementam aplicaÃÃes de alto desempenho. A proposta deste trabalho vai alÃm da construÃÃo de um simples encapsulamento da biblioteca em um componente, visa proporcionar ao uso das bibliotecas cientÃcas os benefÃcios de integraÃÃo, de produtividade em aplicaÃÃes de larga escala e da exibilidade para mudanÃas no contexto do problema. Como forma de exemplicar e validar o mÃtodo, temos incorporado bibliotecas de resoluÃÃo de sistemas lineares ao HPE, elegendo trÃs representantes significativos: PETSc, Hypre e SuperLU. / The contribution of traditional scientic libraries shows to be consolidated in the construction of high-performance applications. However, such an artifact of development possesses some limitations in integration, productivity in large-scale applications, and exibility for changes in the context of the problem. On the other hand, the development technology based on components recently proposed a viable alternative for the architecture of High-Performance Computing (HPC) applications, which has provided a means to overcome these challenges. Thus we see that the scientic libraries and programming orientated at components are complementary techniques in the improvement of the development process of modern HPC applications. Accordingly, this work aims to propose a systematic method for the integration of scientic libraries on a platform of parallel components, HPE (Hash Programming Environment), to oer additional advantageous aspects for the use of components and scientic libraries to developers of parallel programs that implement high-performance applications. The purpose of this work goes beyond the construction of a simple encapsulation of the library in a component; it aims to provide the benets in integration, productivity in large-scale applications, and the exibility for changes in the context of a problem in the use of scientic libraries. As a way to illustrate and validate the method, we have incorporated the libraries of linear systems solvers to HPE, electing three signicant representatives: PETSc, Hypre, e SuperLU.

Orientadores: André Leon Sampaio Gradvohl, Vitor Rafael Coluci / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Tecnologia / Made available in DSpace on 2018-08-21T23:26:09Z (GMT). No. of bitstreams: 1 Andrijauskas_Fabio_M.pdf: 2796809 bytes, checksum: 9fd4e03f6038d482ed05a64517bb1780 (MD5) Previous issue date: 2013 / Resumo: A quantidade de imagens astronômicas geradas cresce diariamente, além da quantidade já obtida e armazenada. Uma grande fonte de dados são imagens solares, cujo estudo pode detectar eventos que têm a capacidade de afetar as telecomunicações, transmissão de energia elétrica e outros sistemas na Terra. Para que tais eventos sejam detectados, torna-se necessário analisar essas imagens de forma eficiente, levando em conta os aspectos de armazenamento, processamento e visualização. Agregar algoritmos de processamento de imagem e técnicas de computação de alto desempenho facilita o tratamento da informação de forma correta e em tempo reduzido. As técnicas de computação para alto desempenho utilizadas neste trabalho foram desenvolvidas para sistemas híbridos, isto é, aqueles que utilizam uma combinação de sistemas de memórias compartilhada e distribuída. Foram produzidas versões paralelas para sistemas híbridos de técnicas já estabelecidas. Além disso, novas técnicas foram propostas e testadas para esse sistema tais como o Filamento Diffusion Detection. Para avaliar a melhora no desempenho, foram feitas comparações entre as versões seriais e paralelas. Esse texto também apresenta um sistema com capacidade para armazenar, processar e visualizar as imagens solares. Em uma das técnicas de detecção de filamentos, o processo foi acelerado 120 vezes e um processo auxiliar para a detecção de áreas mais brilhantes foi 155 vezes mais rápido do que a versão serial / Abstract: The number of astronomical images produced grows daily, in addition to the amount already stored. Great sources of data are solar images, whose study can detect events which have the capacity to affect the telecommunications, electricity transmission and other systems on Earth. For such events being detected, it becomes necessary to treat these images in a coherent way, considering aspects of storage, processing and image visualization. Combining image processing algorithms and high performance computing techniques facilitates the handling of information accurately and in a reduced time. The techniques for high performance computing used in this work were developed for hybrid systems, which employ a combination of shared and distributed memory systems. Parallel version of some established techniques were produced for hybrid systems. Moreover, new techniques have been proposed and tested for this system. To evaluate the improvement in performance, comparisons were made between serial and parallel versions. In addition to the analysis, this text also presents a system with capacity to store, process and visualize solar images. In one of the techniques for detecting filaments, the process was accelerated 120 times. Also an auxiliary process for the detection of brighter areas was 155 times faster than the serial version / Mestrado / Tecnologia e Inovação / Mestre em Tecnologia

The evaluation of football players during and after a football match is an important field of study for football trainers and also people in academia. In this project some of the achieved theoretical results are implemented. Scientific findings are put into a context where the broader general public is able to understand the research. The ultimate goal is to produce visualizations such that they can be integrated into an application called Twelve where an everyday user can access them.