Durant la última dècada, la demanda d’emmagatzematge de dades ha anat creixent exponencialment any rere any. Apart de demanar més capacitat d’emmagatzematge, el usuaris actualment també demanen poder accedir a les seves dades des de qualsevol lloc i des de qualsevol dispositiu. Degut a aquests nous requeriments, els usuaris estan actualment movent les seves dades personals (correus electrònics, documents, fotografies, etc.) cap a serveis d’emmagatzematge en línia com ara Gmail, Facebook, Flickr o Dropbox. Malauradament, aquests serveis d’emmagatzematge en línia estan sostinguts per unes grans infraestructures informàtiques que poques empreses poden finançar. Per tal de reduir el costs d’aquestes grans infraestructures informàtiques, ha sorgit una nova onada de serveis d’emmagatzematge en línia que obtenen grans infraestructures d’emmagatzematge a base d’integrar els recursos petits centres de dades, o fins i tot a base d’integrar els recursos d’emmagatzematge del usuaris finals. No obstant això, els recursos que formen aquestes noves infraestructures d’emmagatzematge són molt heterogenis, cosa que planteja un repte per al dissenyadors d’aquests sistemes: Com es poden dissenyar sistemes d’emmagatzematge en línia, fiables i eficients, quan la infraestructura emprada és tan heterogènia? Aquesta tesis presenta un estudi dels principals problemes que sorgeixen quan un vol respondre a aquesta pregunta. A més proporciona diferents eines per tal d’optimitzar el disseny de sistemes d’emmagatzematge distribuïts i heterogenis. Les principals contribucions són: Primer, creem un marc d’anàlisis per estudiar els efectes de la redundància de dades en el cost dels sistemes d’emmagatzematge distribuïts. Donat un esquema de redundància específic, el marc d’anàlisis presentat permet predir el cost mitjà d’emmagatzematge i el cost mitjà de comunicació d’un sistema d’emmagatzematge implementat sobre qualsevol infraestructura informàtica distribuïda. Segon, analitzem els impactes que la redundància de dades té en la disponibilitat de les dades, i en els temps de recuperació. Donada una redundància, i donat un sistema d’emmagatzematge heterogeni, creem un grup d’algorismes per a determinar la disponibilitat de les dades esperada, i els temps de recuperació esperats. Tercer, dissenyem diferents polítiques d’assignació de dades per a diferents sistemes d’emmagatzematge. Diferenciem entre aquells escenaris on la totalitat de la infraestructura està administrada per una sola organització, i els escenaris on diferents parts auto administrades contribueixen els seus recursos. Els objectius de les nostres polítiques d’assignació de dades són: (i) minimitzar la redundància necessària, (ii) garantir la equitat entre totes les parts que participen al sistema, i (iii) incentivar a les parts perquè contribueixin els seus recursos al sistema. / Over the last decade, users’ storage demands have been growing exponentially year over year. Besides demanding more storage capacity and more data reliability, today users also demand the possibility to access their data from any location and from any device. These new needs encourage users to move their personal data (e.g., E-mails, documents, pictures, etc.) to online storage services such as Gmail, Facebook, Flickr or Dropbox. Unfortunately, these online storage services are built upon expensive large datacenters that only a few big enterprises can afford. To reduce the costs of these large datacenters, a new wave of online storage services has recently emerged integrating storage resources from different small datacenters, or even integrating user storage resources into the provider’s storage infrastructure. However, the storage resources that compose these new storage infrastructures are highly heterogeneous, which poses a challenging problem to storage systems designers: How to design reliable and efficient distributed storage systems over heterogeneous storage infrastructures? This thesis provides an analysis of the main problems that arise when one aims to answer this question. Besides that, this thesis provides different tools to optimize the design of heterogeneous distributed storage systems. The contribution of this thesis is threefold: First, we provide a novel framework to analyze the effects that data redundancy has on the storage and communication costs of distributed storage systems. Given a generic redundancy scheme, the presented framework can predict the average storage costs and the average communication costs of a storage system deployed over a specific storage infrastructure. Second, we analyze the impacts that data redundancy has on data availability and retrieval times. For a given redundancy and a heterogeneous storage infrastructure, we provide a set of algorithms that allow to determine the expected data availability and expected retrieval times. Third, we design different data assignment policies for different storage scenarios. We differentiate between scenarios where the entire storage infrastructure is managed by the same organization, and scenarios where different parties contribute their storage resources. The aims of our assignment policies are: (i) to minimize the required redundancy, (ii) to guarantee fairness among all parties, and (iii) to encourage different parties to contribute their local storage resources to the system.

Šiame darbe nagrinėjamos išskirstytos sistemos ir jų kūrimui naudojamos technologijos. Kadangi technologijų yra daug, o visas jas ištirti ir palyginti būtų labai sunku, todėl didžiausias dėmesys skiriamas Java RMI ir .Net Remoting – tai dvi populiarios išskirstytų objektų paradigmos šiuo metu naudojamos programų sistemų kūrimui Pagrindinis darbo tikslas yra išanalizuoti ir visapusiškai palyginti pasirinktas technologijas, didelį dėmesį skiriant jų veikimo greičiui. Darbas sudarytas iš keturių dalių. Pirmoje dalyje pateikiama bendra informacija apie išskirstytas sistemas, jų vystymąsi, architektūras, technologijų raidą ir t.t.. Antroje dalyje nagrinėjama Java RMI ir .Net Remoting veikimo principai, kuriais remiasi šios technologijos. Trečioje dalyje pateikiami šių dviejų technologijų veikimo principų panašumai ir skirtumai. Ir galiausiai siekiant išsiaiškinti Java RMI ar .Net Remoting realiomis sąlygomis veikia greičiau, ketvirtoje dalyje pateikiamas atlikto vykdymo greičio tyrimo aprašymas. Atlikti darbai leidžia teigti, kad Java RMI ir .Net Remoting idėjiškai yra panašios technologijos, bet skirtumų, įtakojančių veikimą, tarp jų irgi yra. Greičio tyrimo metu gauti rezultatai parodė, kad .Net Remoting beveik visais atvejais yra lėtesnė nei Java RMI. / Distributed systems and technologies used for this type of software are analyzed in this paper. Since there are a lot of technologies and it would be very difficult to explore and compare all of them, therefore this paper mainly focuses on Java and RMI. Net Remoting - two popular distributed objects paradigms currently used in distributed systems development. The first part provides general information about distributed systems, their development, architectures, technologies, and so on. The second part of the paper analyses the operating principles of Java RMI and. Net Remoting. The third section determines technological similarities and differencies. And finally, in order to clarify the fact Java RMI or. Net Remoting works faster in real terms, the fourth part provides the results of the implemented speed test. Although Java RMI and. Net Remoting are similar technologies, but the analysis showed that the there are quite a lot differences between them. The speed survey showed that. Net Remoting is slower than Java RMI in almost all cases.

Adaptive distributed systems have the potential to revolutionize how humans and computers interact. They can enable software to adapt to dynamic human demands, as users change their focuses, goals, locations and devices. However, producing these systems is currently very challenging: developers must solve daunting user interface issues while mired in distributed systems problems. We present a new class of toolkit, intended to ease the development of adaptive distributed systems. Unlike existing alternatives, we provide a high-level programming model in which developers can easily specify runtime adaptations. Meanwhile, our toolkit automatically generates a fast and tunable implementation. Partial failures in the distributed system are reflected back into the high-level programming model. As a result, developers can remain insulated in their high-level model while building highly-dynamic, high-performance and failure-resistant applications. Our Fiia.Net toolkit relies on model transformation at runtime to bridge between the programmer's high-level model and the actual implementation of the distributed system. Our novel model transformation is the first that can practically maintain this transformation, and enables our toolkit to easily support user-driven adaptations, dynamic optimization and self-healing. / Thesis (Ph.D, Computing) -- Queen's University, 2011-02-25 14:08:13.69

Publish/Subscribe (in short pub/sub) allows clients that share common interest communicate in an asynchronous and loosely-coupled fashion. This paradigm is adopted by many distributed event-driven applications such as social networking services, distributed business processes and cyber-physical systems. These applications cannot afford to have the underlying pub/sub substrate perform unreliably, permanently fail or behave arbitrarily as it will cause significant disturbance to stably serving many end-users. Therefore, a research effort on making pub/sub systems resilient against various failures to sustain high quality of service to the clients is imperative. In this thesis, we focus on the overlay of pub/sub brokers that are widely adopted as a popular architecture for large-scale pub/sub systems. Broker overlays can suffer from various issues such as degradation of topology quality, brokers causing transient or permanent benign failures and Byzantine brokers behaving arbitrarily. We aim to make novel research contributions by exploring fundamental techniques that can help the broker overlays maintain functional and non-functional requirements even under the presence of the aforementioned failures and necessary administrative updates. We first build a set of overlay adaptation primitives that re-configure topologies such as shifting links and replicating brokers. These primitives are designed to involve a small local group of brokers in the pub/sub overlays so that the disruption during the execution of large-scale and dynamic changes can be controlled in a fined-grained manner. For the problem of degrading topology quality, automated planning systems are developed to find a sequence of adaptations that would cause minimal disruption to running services. Also, our primitives can be executed on demand to quickly fail-over a crashed broker or off-load congested brokers. In addition, these on-demand primitives can be used to form a group of dynamically replicated brokers that enforce a novel safety measure to prevent Byzantine brokers from sabotaging the pub/sub overlays. Our contributions are evaluated with systematic consideration of various trade-offs between functional and non-functional properties.

Publish/Subscribe (in short pub/sub) allows clients that share common interest communicate in an asynchronous and loosely-coupled fashion. This paradigm is adopted by many distributed event-driven applications such as social networking services, distributed business processes and cyber-physical systems. These applications cannot afford to have the underlying pub/sub substrate perform unreliably, permanently fail or behave arbitrarily as it will cause significant disturbance to stably serving many end-users. Therefore, a research effort on making pub/sub systems resilient against various failures to sustain high quality of service to the clients is imperative. In this thesis, we focus on the overlay of pub/sub brokers that are widely adopted as a popular architecture for large-scale pub/sub systems. Broker overlays can suffer from various issues such as degradation of topology quality, brokers causing transient or permanent benign failures and Byzantine brokers behaving arbitrarily. We aim to make novel research contributions by exploring fundamental techniques that can help the broker overlays maintain functional and non-functional requirements even under the presence of the aforementioned failures and necessary administrative updates. We first build a set of overlay adaptation primitives that re-configure topologies such as shifting links and replicating brokers. These primitives are designed to involve a small local group of brokers in the pub/sub overlays so that the disruption during the execution of large-scale and dynamic changes can be controlled in a fined-grained manner. For the problem of degrading topology quality, automated planning systems are developed to find a sequence of adaptations that would cause minimal disruption to running services. Also, our primitives can be executed on demand to quickly fail-over a crashed broker or off-load congested brokers. In addition, these on-demand primitives can be used to form a group of dynamically replicated brokers that enforce a novel safety measure to prevent Byzantine brokers from sabotaging the pub/sub overlays. Our contributions are evaluated with systematic consideration of various trade-offs between functional and non-functional properties.

abstract: Peer-to-peer systems are known to be vulnerable to the Sybil attack. The lack of a central authority allows a malicious user to create many fake identities (called Sybil nodes) pretending to be independent honest nodes. The goal of the malicious user is to influence the system on his/her behalf. In order to detect the Sybil nodes and prevent the attack, a reputation system is used for the nodes, built through observing its interactions with its peers. The construction makes every node a part of a distributed authority that keeps records on the reputation and behavior of the nodes. Records of interactions between nodes are broadcast by the interacting nodes and honest reporting proves to be a Nash Equilibrium for correct (non-Sybil) nodes. In this research is argued that in realistic communication schedule scenarios, simple graph-theoretic queries such as the computation of Strongly Connected Components and Densest Subgraphs, help in exposing those nodes most likely to be Sybil, which are then proved to be Sybil or not through a direct test executed by some peers. / Dissertation/Thesis / Ph.D. Computer Science 2010

Este trabalho apresenta um modelo de integração de ferramentas de animação que reduz consideravelmente o esforço envolvido na construção de novos sistemas. O modelo proposto distribui as tarefas a serem implementadas pelo sistema entre diversas ferramentas, permitindo que cada uma seja menor e mais fácil de manter que um sistema completo. Uma implementação do modelo proposto também é apresentada aqui, assim como o é um protótipo de sistema, construído de acordo com o modelo. O protótipo é comparado com as ferramentas de animação atualmente disponíveis na UFRGS, a universidade na qual este estudo foi desenvolvido. O trabalho conclui com uma análise dos resultados principais obtidos deste estudo. Ao final também são apresentadas algumas sugestões de trabalhos futuros. / This work presents a model for animation tool’s integration, which greatly reduces the programmer’s work in developing a new system. The proposed model distributes all tasks among several tools, making each tool smaller and easier to maintain. An implementation of such model is also described here, and so is a prototype of an animation system made according to the model proposed. The prototype is then compared to the current animation tools available at the UFRGS, the University in which this work was developed. The work concludes with an analysis of the main results and some suggestions of possible future works.

Durante os últimos anos, a quantidade de dados gerada pelas aplicações cresceu consideravelmente. No entanto, para tornarem-se relevantes estes dados precisam ser processados. Para atender este objetivo, novos modelos de programação para processamento paralelo e distribuído foram propostos. Um exemplo é o modelo MapReduce, o qual foi proposto pela Google. Este modelo, no entanto, possui pontos de falha única (SPOF), os quais podem comprometer a sua execução. Assim, este trabalho apresenta uma nova arquitetura, inspirada pelo Chord, para lidar com os SPOFs do modelo. A avaliação da proposta foi realizada através de modelagem analítica e de testes experimentais. Os resultados mostram a viabilidade de usar a arquitetura proposta para executar o MapReduce. / During the last years, the amount of data generated by applications grew considerably. To become relevant, however, this data should be processed. With this goal, new programming models for parallel and distributed processing were proposed. An example is the MapReduce model, which was proposed by Google. This model, nevertheless, has Single Points of Failure (SPOF), which can compromise the execution of a job. Thus, this work presents a new architecture, inspired by Chord, to avoid the SPOFs on MapReduce. The evaluation was performed through an analytical model and an experimental setup. The results show the feasibility of using the proposed architecture to execute MapReduce jobs.

FURTADO JÚNIOR, C. G. Contribuições à modelagem e à simulação de sistemas distribuídos usando redes de Petri coloridas - alocação de webcaches e particionamento de modelos em processos independentes. 2016. 109 f. Tese (Doutorado em Engenharia de Teleinformática)–Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza, 2016. / Submitted by Marlene Sousa (mmarlene@ufc.br) on 2017-08-22T18:04:15Z No. of bitstreams: 1 2016_tese_cgfurtadojunior.pdf: 8950611 bytes, checksum: 75456ddc606a7b3205e18456280b733a (MD5) / Approved for entry into archive by Marlene Sousa (mmarlene@ufc.br) on 2017-08-22T18:05:13Z (GMT) No. of bitstreams: 1 2016_tese_cgfurtadojunior.pdf: 8950611 bytes, checksum: 75456ddc606a7b3205e18456280b733a (MD5) / Made available in DSpace on 2017-08-22T18:05:13Z (GMT). No. of bitstreams: 1 2016_tese_cgfurtadojunior.pdf: 8950611 bytes, checksum: 75456ddc606a7b3205e18456280b733a (MD5) Previous issue date: 2016-12-02 / The investigation of specific aspects of a system through simulations makes it possible to identify in advance faults, blocks and instabilities , among other aspects. Thus, the costs and risks associated with the implementation of systems are reduced, making it possible to analyze the projects of these systems in different scenarios. To this end, coloured Petri nets (CPN) are often used. However, simulating systems at multiple levels of abstraction, including specific analyzes and performance evaluations, may require highly specialized technical knowledge, increasing the cost of the simulation. In addition, simulators developed for large systems that allow complex configurations may require a large amount of computational resources, compromising simulations based on CPN. Aiming to provide mechanisms to reduce the cost of extensibility and increase the robustness of simulations performed in CPN, two contributions are presented in this thesis. In the first one, we present the Multi-Language Programming Interface (IPML), which allows to extend simulators without in the CPN. In order to validate IPML, two extensible simulators were developed: CacheSim, a tool that provides a platform for modeling, configuration and use of Internet caching policies, and TransportSim, a configurable tool that allows the representation and analysis of simulation scenarios for metro / rail transportation. It is demonstrated, in the form of use cases, how to extend these simulators without modifying the CPN. As a second contribution, we propose distributed colored Petri nets (DCPNs), which allow the design of individual models that are not necessarily hierarchical, and which can be executed in independent processes on different computers. The distribution of a CPN model is performed with the addition of the Distributed Fusion Place concept. Simulations are possible with the execution of algorithms that allow the communication and coordination of the multiple processes that contain the individual models and execute simultaneously. In order to dispense with the use of priorities as a solution to distributed conflicts and thus not to compromise the analytical power of the CPN, a protocol has been created for simultaneous access to multiple critical regions, ensuring fairness in the selection of events that may occur and allowing parallel execution of non-conflicting transitions. The DCPN allows the conception and distributed simulation of CPN without adding new semantic resources to CPN. This allows the transformation of centralized simulators into a single process in simulators that can be distributed in multiple execution nodes, with effective low development cost. The formal definition of DCPN is presented and the analysis of a distributed model is discussed. The results demonstrate that the addition of new processes increases the performance of a simulation up to a saturation limit, in which the cost of communication between the multiple nodes does not compensate the segmentation of the model. / A investigação de aspectos específicos de um sistema computacional por meio de simulações possibilita identificar antecipadamente falhas, bloqueios e instabilidades, entre outros aspectos. Reduzem-se, assim, os custos e os riscos associados à implementação de sistemas, viabilizandose a análise de projetos desses sistemas em diferentes cenários. Com esta finalidade, as Redes de Petri coloridas (RPC) vêm sendo frequentemente utilizadas. Entretanto, simular sistemas em múltiplos níveis de abstração, incluindo análises específicas e avaliações de desempenho, podem requerer conhecimentos técnicos altamente especializados, aumentando o custo do próprio trabalho de simulação. Adicionalmente, simuladores desenvolvidos para grandes sistemas e que permitam configurações complexas podem exigir elevada quantidade de recursos computacionais, comprometendo simulações realizadas por meio de RPC. Visando fornecer mecanismos que permitam reduzir o custo da extensibilidade e aumentar a robustez de simuações realizadas em RPC, são apresentadas neste trabalho duas contribuições. Na primeira, apresentamos a Interface de Programação Multilinguagem (IPML), que permite a extensão de simuladores construídos por RPC sem modificações na rede. Para validar a IPML, foram desenvolvidos dois simuladores extensíveis, o CacheSim, ferramenta que disponibiliza uma plataforma para modelagem, configuração e uso de políticas de substituição de arquivos em cache da Internet, e o TransportSim, uma ferramenta configurável que permite a representação e análise de cenários de simulação para transportes metroviários/ferroviários. Demonstra-se, em forma de casos de uso, como estender esses simuladores sem a necessidade de modificação das RPC. Como segunda contribuição, propomos as redes de Petri coloridas distribuídas (RPCD), que permitem a concepção de modelos em sub-partes não necessariamente hierárquicas, e que podem executar em processos independentes em diferentes computadores. A distribuição de um modelo RPC é realizada com a adição do conceito de Lugar de Fusão Distribuído. As simulações são possíveis com a execução de algoritmos que permitem a comunicação e a coordenação dos múltiplos processos que contêm as sub-partes e executam simultaneamente. De maneira a prescindir do uso de prioridades como solução para conflitos distribuídos e, dessa maneira, não comprometer o poder analítico das RPC, foi criado um protocolo para acesso simultâneo a múltiplas regiões críticas, garantindo justiça na seleção dos eventos que podem ocorrer e permitindo a execução paralela de transições não conflitantes. As RPCD permitem a concepção e simulação distribuída de RPC sem adição de novos recursos semânticos às RPCs. Isso possibilita a transformação de simuladores centralizados num único processo em simuladores que podem ser distribuídos em múltiplos nós de execução, com baixo custo efetivo de desenvolvimento. A definição formal de RPCD é apresentada e a análise de um modelo distribuído é discutida. Os resultados indicam que a adição de novos processos incrementa o desempenho , em termos de velocidade, de uma simulação até um limite de saturação, em que o custo de comunicação entre os múltiplos nós passa a não compensar a segmentaçao do modelo.

O surgimento de volumes de dados na ordem de petabytes cria a necessidade de desenvolver-se novas soluções que viabilizem o tratamento dos dados através do uso de sistemas de computação intensiva, como o MapReduce. O MapReduce é um framework de programação que apresenta duas funções: uma de mapeamento, chamada Map, e outra de redução, chamada Reduce, aplicadas a uma determinada entrada de dados. Este modelo de programação é utilizado geralmente em grandes clusters e suas tarefas Map ou Reduce são normalmente independentes entre si. O programador é abstraído do processo de paralelização como divisão e distribuição de dados, tolerância a falhas, persistência de dados e distribuição de tarefas. A motivação deste trabalho é aplicar o modelo de computação intensiva do MapReduce com grande volume de dados para uso em ambientes desktop grid. O objetivo então é investigar os algoritmos do MapReduce para adequar a computação intensiva aos ambientes heterogêneos. O trabalho endereça o problema da heterogeneidade de recursos, não tratando neste momento a volatilidade das máquinas. Devido às deficiências encontradas no MapReduce em ambientes heterogêneos foi proposto o MR-A++, que é um MapReduce com algoritmos adequados ao ambiente heterogêneo. O modelo do MR-A++ cria uma tarefa de medição para coletar informações, antes de ocorrer a distribuição dos dados. Assim, as informações serão utilizadas para gerenciar o sistema. Para avaliar os algoritmos alterados foi empregada a Análise 2k Fatorial e foram executadas simulações com o simulador MRSG. O simulador MRSG foi construído para o estudo de ambientes (homogêneos e heterogêneos) em larga escala com uso do MapReduce. O pequeno atraso introduzido na fase de setup da computação é compensado com a adequação do ambiente heterogêneo à capacidade computacional das máquinas, com ganhos de redução de tempo de execução dos jobs superiores a 70 % em alguns casos. / The emergence of data volumes in the order of petabytes creates the need to develop new solutions that make possible the processing of data through the use of intensive computing systems, as MapReduce. MapReduce is a programming framework that has two functions: one called Map, mapping, and another reducing called Reduce, applied to a particular data entry. This programming model is used primarily in large clusters and their tasks are normally independent. The programmer is abstracted from the parallelization process such as division and data distribution, fault tolerance, data persistence and distribution of tasks. The motivation of this work is to apply the intensive computation model of MapReduce with large volume of data in desktop grid environments. The goal then is to investigate the intensive computing in heterogeneous environments with use MapReduce model. First the problem of resource heterogeneity is solved, not treating the moment of the volatility. Due to deficiencies of the MapReduce model in heterogeneous environments it was proposed the MR-A++; a MapReduce with algorithms adequated to heterogeneous environments. The MR-A++ model creates a training task to gather information prior to the distribution of data. Therefore the information will be used to manager the system. To evaluate the algorithms change it was employed a 2k Factorial analysis and simulations with the simulant MRSG built for the study of environments (homogeneous and heterogeneous) large-scale use of MapReduce. The small delay introduced in phase of setup of computing compensates with the adequacy of heterogeneous environment to computational capacity of the machines, with gains in the run-time reduction of jobs exceeding 70% in some cases.