<p>We study techniques to enforce and provision differentiated service levels in <i>Computational Grid</i> systems. The Grid offers simplified provisioning of peak-capacity for applications with computational requirements beyond local machines and clusters, by sharing resources across organizational boundaries. Current systems have focussed on access control, i.e., managing who is allowed to run applications on remote sites. Very little work has been done on providing differentiated service levels for those applications that are admitted. This leads to a number of problems when scheduling jobs in a fair and efficient way. For example, users with a large number of long-running jobs could starve out others, both intentionally and non-intentionally. We investigate the requirements of High Performance Computing (HPC) applications that run in academic Grid systems, and propose two models of service-level management. Our first model is based on global real-time quota enforcement, where projects are granted resource quota, such as CPU hours, across the Grid by a centralized allocation authority. We implement the SweGrid Accounting System to enforce quota allocated by the Swedish National Allocations Committee in the SweGrid production Grid, which connects six Swedish HPC centers. A flexible authorization policy framework allows provisioning and enforcement of two different service levels across the SweGrid clusters; high-priority and low-priority jobs. As a solution to more fine-grained control over service levels we propose and implement a <i>Grid</i> <i>Market </i>system, using a market-based resource allocator called Tycoon. The conclusion of our research is that although the Grid accounting solution offers better service level enforcement support than state-of-the-art production Grid systems, it turned out to be complex to set the resource price and other policies manually, while ensuring fairness and efficiency of the system. Our Grid Market on the other hand sets the price according to the dynamic demand, and it is further incentive compatible, in that the overall system state remains healthy even in the presence of strategic users.</p>

We study techniques to enforce and provision differentiated service levels in Computational Grid systems. The Grid offers simplified provisioning of peak-capacity for applications with computational requirements beyond local machines and clusters, by sharing resources across organizational boundaries. Current systems have focussed on access control, i.e., managing who is allowed to run applications on remote sites. Very little work has been done on providing differentiated service levels for those applications that are admitted. This leads to a number of problems when scheduling jobs in a fair and efficient way. For example, users with a large number of long-running jobs could starve out others, both intentionally and non-intentionally. We investigate the requirements of High Performance Computing (HPC) applications that run in academic Grid systems, and propose two models of service-level management. Our first model is based on global real-time quota enforcement, where projects are granted resource quota, such as CPU hours, across the Grid by a centralized allocation authority. We implement the SweGrid Accounting System to enforce quota allocated by the Swedish National Allocations Committee in the SweGrid production Grid, which connects six Swedish HPC centers. A flexible authorization policy framework allows provisioning and enforcement of two different service levels across the SweGrid clusters; high-priority and low-priority jobs. As a solution to more fine-grained control over service levels we propose and implement a Grid Market system, using a market-based resource allocator called Tycoon. The conclusion of our research is that although the Grid accounting solution offers better service level enforcement support than state-of-the-art production Grid systems, it turned out to be complex to set the resource price and other policies manually, while ensuring fairness and efficiency of the system. Our Grid Market on the other hand sets the price according to the dynamic demand, and it is further incentive compatible, in that the overall system state remains healthy even in the presence of strategic users. / QC 20101116

Made available in DSpace on 2014-07-29T14:57:51Z (GMT). No. of bitstreams: 1 Inicias_Mestrado_Jesus_Neto.pdf: 214553 bytes, checksum: 20f491ce03a3808c0630f4ac941f35af (MD5) Previous issue date: 2008-04-25 / Computer grids are sets of computational resources that provide diverse types of services, such as storage and processing, on behalf of applications spread across different administrative domains. Many companies and academic institutions have demonstrated interest in their use for the execution of applications that demand huge amounts of computation power and storage. However, computer grids are complex and diversified execution environments, which exhibit high variation of resource availability, node instability and variations on load distribution, among other problems. This work presents a model of dynamic interceptors and its use in InteGrade, an opportunistic grid middleware. The use of interceptors aims to provide dynamic adaptation in InteGrade through its communication middleware, thus contributing to make InteGrade able to deal with the highly variable execution environment of computer grids without requiring changes to its implementation. Therefore, this work aims to offer dynamic adaptation capabilities to InteGrade and not to grid applications. Nevertheless, these applications will be able to benefit from adaptation provided by InteGrade / Grades computacionais são conjuntos de recursos computacionais que fornecem diversos tipos de serviços, tais como armazenamento e processamento, para aplicações que podem estar espalhadas por diferentes domínios administrativos. Desta forma, várias empresas e instituições acadêmicas têm interesse no seu uso para a execução de aplicações que exijam um alto poder computacional. Entretanto, grades computacionais são ambientes de execução bastante diversificados e complexos, pois possuem alta variação na disponibilidade de recursos, instabilidade de seus nós e variações na distribuição de carga, entre outros problemas. Este trabalho apresenta um modelo de interceptadores dinâmicos e seu uso no InteGrade, um middleware de grade oportunista. O uso de interceptadores tem por finalidade prover suporte para adaptação dinâmica no InteGrade através de seu middleware de comunicação, assim, contribuindo para que o mesmo tenha condições de lidar com o ambiente de execução altamente variável das grades computacionais sem que sejam necessárias alterações em sua implementação. Desta forma, este trabalho busca fornecer recursos de adaptação dinâmica para o InteGrade e não para aplicações de grade. No entanto, estas aplicações poderão se beneficiar dos recursos de adaptação oferecidos pelo InteGrade

Climate science or climatology is the scientific study of the earth’s climate, where climate is the term representing weather conditions averaged over a period of time. Climate models are mathematical models used to quantitatively describe, simulate and study the interactions among the components of the climate system -atmosphere, ocean, land and sea-ice. CCSM (Community Climate System Model) is a state-of-the-art climate model, and a long-running coupled multicomponent parallel application involving component models for simulating the components of the climate system. Each of the component models is a large-scale parallel application, and the parallel components exchange climate data through a specialized component called coupler. Typical multi-century climate simulations using CCSM take several weeks or months to execute on most parallel systems. In this thesis, we study the applicability of a computational grid for effective execution of long-running coupled multi-component climate applications like CCSM. Initial studies of the application characteristics led us to develop a dynamic component extension strategy for temporal inter-component load-balancing. By means of experiments on different parallel platforms with different number of processors, we showed that using our strategy can lead to about 15% reduction and savings of several days in execution times of CCSM for 1000-year simulation runs. Our initial studies also indicated that unlike typical grid applications, CCSM has limits on scalability to very large number of processors and hence cannot directly benefit from the large number of processors on a computational grid. However, its long-running nature and the limits of execution imposed on jobs on most multi-user batch queueing systems, led us to investigate the benefits of its execution on a grid of batch systems. The idea is that multiple batch queues can improve the processor availability rate with respect to the application thereby possibly improving its effective throughput. We explored this idea in detail with simulation studies involving various system and application characteristics, and execution models. By conducting large number of simulations with different workload characteristics and queuing policies of the systems, processor allocations to components of the application, distributions of the components to the batch systems and inter-cluster bandwidths, we showed that multiple batch executions lead to upto 55% average increase in throughput over single batch executions for long-running CCSM. Having convinced ourselves of possible advantages in performance, we then ventured to construct an application-level middleware framework. Our framework supports long duration execution of multi-component applications spanning multiple submissions to queues on multiple batch systems. It coordinates the distribution, execution, rescheduling, migration and restart of the application components across resources on different sites. It also addresses challenges including execution time limits for jobs, and differences in job-startup times corresponding to different components. Further, within the framework, we developed robust rescheduling policies that decide when and where to reschedule the components to the available resources based on the application execution characteristics and queue dynamics. Our grid middleware framework resulted in multi-site executions that provided larger application throughput than single-site executions, typically performed by climate scientists, and also removed the bottlenecks associated with a single system execution. We used this framework for long-running executions of CCSM to study the effect of increased black carbon aerosols and dust aerosols on the Indian monsoons. Black Carbon aerosols are essentially of anthropogenic origin and occur due to improper burning of fossil fuels, and dust is a naturally occurring aerosol. The concentrations of both these aerosols is high over the Indian region. We study the impact of these aerosols on precipitation and sea surface temperature (SST) through multi-decadal simulations conducted with our grid-enabled climate system model. Our observations indicated that increasing the concentrations of aerosols leads to an increase in precipitation in the central and eastern parts of India, and a decrease in SST over most of Indian ocean.