Global ETD Search

11	Otimização de operações de entrada e saída visando reduzir o tempo de resposta de aplicações distribuídas que manipulam grandes volumes de dados / Optimization input output operations aiming at reduce execution time of distributed applications which handle large amount of data Renato Porfirio Ishii 01 September 2010 (has links) Aplicações científicas atuais têm produzido volumes de dados cada vez maiores. O processamento, a manipulação e a análise desses dados requerem infraestruturas computacionais de larga escala tais como aglomerados e grades de computadores. Nesse contexto, várias pesquisas visam o aumento de desempenho dessas aplicações por meio da otimização de acesso a dados. Para alcançar tal objetivo, pesquisadores têm utilizado técnicas de replicação, migração, distribuição e paralelismo de dados. No entanto, uma das principais lacunas dessas pesquisas está na falta de emprego de conhecimento sobre aplicações com objetivo de realizar essa otimização. Essa lacuna motivou esta tese que visa empregar comportamento histórico e preditivo de aplicações a fim de otimizar suas operações de leitura e escrita sobre dados distribuídos. Os estudos foram iniciados empregando-se informações previamente monitoradas de aplicações a fim de tomar decisões relativas à replicação, migração e manutenção de consistência. Observou-se, por meio de uma nova heurística, que um conjunto histórico de eventos auxilia a estimar o comportamento futuro de uma aplicação e otimizar seus acessos. Essa primeira abordagem requer ao menos uma execução prévia da aplicação para composição de histórico. Esse requisito pode limitar aplicações reais que apresentam mudanças comportamentais ou que necessitam de longos períodos de execução para completar seu processamento. Para superar essa limitação, uma segunda abordagem foi proposta baseada na predição on-line de eventos comportamentais de aplicações. Essa abordagem não requer a execução prévia da aplicação e permite adaptar estimativas de comportamento futuro em função de alterações adjacentes. A abordagem preditiva analisa propriedades de séries temporais com objetivo de classificar seus processos geradores. Essa classificação aponta modelos que melhor se ajustam ao comportamento das aplicações e que, portanto, permitem predições com maior acurácia. As duas abordagens propostas foram implementadas e avaliadas utilizando o simulador OptorSim, vinculado ao projeto LHC/CERN, amplamente adotado pela comunidade científica. Experimentos constataram que as duas abordagens propostas reduzem o tempo de resposta (ou execução) de aplicações que manipulam grandes volumes de dados distribuídos em aproximadamente 50% / Current scientific applications produce large amount of data and handling, processing and analyzing such data require large-scale computing infrastructure such as clusters and grids. In this context, various studies have focused at improving the performance of these applications by optimizing data access. In order to achieve this goal, researchers have employed techniques of replication, migration, distribution and parallelism of data. However, these common approaches do not use knowledge about the applications at hand to perform this optimization. This gap motivated the present thesis, which aims at applying historical and predictive behavior of applications to optimize their reading and writing operations on distributed data. Based on information previously monitored from applications to make decisions regarding replication, migration and consistency of data, a new heuristic was initially proposed. Its evaluation revealed that considering sets of historical events indeed helps to estimate the behavior of future applications and to optimize their access operations. Thus it was embedded into two optimization approaches. The first one requires at least a previous execution for the history composition. This requirement may limit real world applications which present behavioral changes or take very long time to execute. In order to overcome this issue, a second technique was proposed. It performs on-line predictions about the behavior of the applications, mitigating the need of any prior execution. Additionally, this approach considers the future behavior of an application as a function of its underlying changes. This behavior can be modeled as time series. The method works by analyzing the series properties in order to classify their generating processes. This classification indicates models that best fit the applications behavior, allowing more accurate predictions. Experiments using the OptorSim simulator (LHC/CERN project) confirmed that the proposed approaches are able to reduce the response time of applications that handle large amount of distributed data in approximately 50% Análise de séries temporais Computação distribuída Otimização de acesso a dados Sistemas de arquivos distribuídos Data access optimization Distributed computing Distributed file system Time series analysis
12	Serverová část systému pro správu projektové dokumentace / Server Part of the Project Documentation Management System Černobila, Radek January 2008 (has links) The goal of this work is proposal and implementation of the revision system that can control whole development process of a project. It shows alternatives of the implementation of its parts. The main is a persistent data storage and the system architecture. The expected result is a application that is able to present basic solved problems as are project branching and distributed data storage.
13	Darknet file sharing : application of a private peer-to-peer distributed file system concept Ledung, Gabriel, Andersson, Johan January 2010 (has links) Peer-to-peer network applications has been a tremendous success among end users and has therefore received much attention in academia and industry, as have illegal public file sharing in media. However, private peer-to-peer file sharing between family, friends and co-workers have attracted little interest from the research community. Existing approaches also limit the users by not allowing for native interaction with userspace applications. In this paper we ex- -plore how private file sharing can be made safe, fast and scalable without constraining the users in this aspect. We demonstrate the concept of a private file sharing application utilizing a decentralized peer-to-peer network overlay by creating a prototype with extreme program- ming as methodology. To maximize the freedom of users the network is accessed through a virtual file-system interface. The prototype proves this to be a valid approach and we hope readers can use this paper as a platform for further developments in this area. / Fildelningsapplikationer som använder peer-to-peer teknik har varit en enorm framgång blandslutanvändare och har därmed erhållit mycket uppmärksamhet från akademi och indus- tri, liksom olaglig fildelning fått inom media. Däremot har inte privat fildelning mellan vän- ner, arbetskamrater och kollegor tilldelats samma uppmärksamhet från forskningssamfundet. Nuvarande tillämpningar begränsar användaren genom att inte tillåta naturlig interaktion med användarapplikationer. I denna uppsats utforskar vi hur privat fildelning kan göras snabb, skalbar och säker utan att begränsa användaren ur den aspekten. Vi demonstrerar ett koncept- för privat fildelning som nyttjar decentraliserad peer-to-peer arkitektur m.h.a en prototyp som tagits fram med extreme programming som metodologi. För att maximera användarnas frihet nyttjas ett virtuellt filsystem som gränssnitt. Prototypen visar att vår tillämpning fungerar i praktiken och vi hoppas att läsaren kan använda vårt arbete som en plattform för fortsatt utveckling inom detta område. file sharing darknet peer-to-peer P2P distributed file system virtual file system anonymity BitTorrent distributed hash tables DHT fildelning annonymitet p2p distribuerade filsystem darknet bittorrent Information Systems
14	Efficient Usage Of Flash Memories In High Performance Scenarios Srimugunthan, * 10 1900 (has links) (PDF) New PCI-e flash cards and SSDs supporting over 100,000 IOPs are now available, with several usecases in the design of a high performance storage system. By using an array of flash chips, arranged in multiple banks, large capacities are achieved. Such multi-banked architecture allow parallel read, write and erase operations. In a raw PCI-e flash card, such parallelism is directly available to the software layer. In addition, the devices have restrictions such as, pages within a block can only be written sequentially. The devices also have larger minimum write sizes (>4KB). Current flash translation layers (FTLs) in Linux are not well suited for such devices due to the high device speeds, architectural restrictions as well as other factors such as high lock contention. We present a FTL for Linux that takes into account the hardware restrictions, that also exploits the parallelism to achieve high speeds. We also consider leveraging the parallelism for garbage collection by scheduling the garbage collection activities on idle banks. We propose and evaluate an adaptive method to vary the amount of garbage collection according to the current I/O load on the device. For large scale distributed storage systems, flash memories are an excellent choice because flash memories consume less power, take lesser floor space for a target throughput and provide faster access to data. In a traditional distributed filesystem, even distribution is required to ensure load-balancing, balanced space utilisation and failure tolerance. In the presence of flash memories, in addition, we should also ensure that the numbers of writes to these different flash storage nodes are evenly distributed, to ensure even wear of flash storage nodes, so that unpredictable failures of storage nodes are avoided. This requires that we distribute updates and do garbage collection, across the flash storage nodes. We have motivated the distributed wearlevelling problem considering the replica placement algorithm for HDFS. Viewing the wearlevelling across flash storage nodes as a distributed co-ordination problem, we present an alternate design, to reduce the message communication cost across participating nodes. We demonstrate the effectiveness of our design through simulation. Flash Cards Flash Memory Scaling High Performance Computing Distributed Flash Memories Word Recognition Flash Translation Layer (FTL) Distributed File System Parallelism Flash Cards Flash Cards - Garbage Collection Flash Storage Nodes Cluster Storage Distributed Storage System Computer Science
15	Zpracování velkých dat z rozsáhlých IoT sítí / Big Data Processing from Large IoT Networks Benkő, Krisztián January 2019 (has links) The goal of this diploma thesis is to design and develop a system for collecting, processing and storing data from large IoT networks. The developed system introduces a complex solution able to process data from various IoT networks using Apache Hadoop ecosystem. The data are real-time processed and stored in a NoSQL database, but the data are also stored in the file system for a potential later processing. The system is optimized and tested using data from IQRF network. The data stored in the NoSQL database are visualized and the system periodically generates derived predictions. Users are connected to this system via an information system, which is able to automatically generate notifications when monitored values are out of range.
16	Sistemas de arquivos paralelos: alternativas para a redução do gargalo no acesso ao sistema de arquivos / Parallel File Systems: alternatives to reduce the bottleneck in accessing the file system Carvalho, Roberto Pires de 23 September 2005 (has links) Nos últimos anos, a evolução dos processadores e redes para computadores de baixo custo foi muito maior se comparada com o aumento do desempenho dos discos de armazenamento de dados. Com isso, muitas aplicações estão encontrando dificuldades em atingir o pleno uso dos processadores, pois estes têm de esperar até que os dados cheguem para serem utilizados. Uma forma popular para resolver esse tipo de empecílio é a adoção de sistemas de arquivos paralelos, que utilizam a velocidade da rede local, além dos recursos de cada máquina, para suprir a deficiência de desempenho no uso isolado de cada disco. Neste estudo, analisamos alguns sistemas de arquivos paralelos e distribuídos, detalhando aqueles mais interessantes e importantes. Por fim, mostramos que o uso de um sistema de arquivos paralelo pode ser mais eficiente e vantajoso que o uso de um sistema de arquivos usual, para apenas um cliente. / In the last years, the evolution of the data processing power and network transmission for low cost computers was much bigger if compared to the increase of the speed of getting the data stored in disks. Therefore, many applications are finding difficulties in reaching the full use of the processors, because they have to wait until the data arrive before using. A popular way to solve this problem is to use a parallel file system, which uses the local network speed to avoid the performance bottleneck found in an isolated disk. In this study, we analyze some parallel and distributed file systems, detailing the most interesting and important ones. Finally, we show the use of a parallel file system can be more efficient than the use of a usual local file system, for just one client. acesso concorrente afs afs availability bridge bridge ceft-pvfs ceft-pvfs coda coda computação distribuída computação paralela concurrent access disco magnético disco rígido disponibilidade distributed computing distributed file system escalabilidade fail over file replication file service file system file system bottleneck gargalo no sistema de arquivos gfs gfs hard disk ide ide latência latency magnetic disk nfs nfs nfsp nfsp parallel computing parallel file system pio pio pvfs pvfs pvfs2 pvfs2 replicação de arquivos sad scalability scsi scsi security segurança serviço de arquivos sistema de arquivos sistema de arquivos distribuídos sistema de arquivos paralelos sprite sprite sva sva tolerância a falhas
17	Sistemas de arquivos paralelos: alternativas para a redução do gargalo no acesso ao sistema de arquivos / Parallel File Systems: alternatives to reduce the bottleneck in accessing the file system Roberto Pires de Carvalho 23 September 2005 (has links) Nos últimos anos, a evolução dos processadores e redes para computadores de baixo custo foi muito maior se comparada com o aumento do desempenho dos discos de armazenamento de dados. Com isso, muitas aplicações estão encontrando dificuldades em atingir o pleno uso dos processadores, pois estes têm de esperar até que os dados cheguem para serem utilizados. Uma forma popular para resolver esse tipo de empecílio é a adoção de sistemas de arquivos paralelos, que utilizam a velocidade da rede local, além dos recursos de cada máquina, para suprir a deficiência de desempenho no uso isolado de cada disco. Neste estudo, analisamos alguns sistemas de arquivos paralelos e distribuídos, detalhando aqueles mais interessantes e importantes. Por fim, mostramos que o uso de um sistema de arquivos paralelo pode ser mais eficiente e vantajoso que o uso de um sistema de arquivos usual, para apenas um cliente. / In the last years, the evolution of the data processing power and network transmission for low cost computers was much bigger if compared to the increase of the speed of getting the data stored in disks. Therefore, many applications are finding difficulties in reaching the full use of the processors, because they have to wait until the data arrive before using. A popular way to solve this problem is to use a parallel file system, which uses the local network speed to avoid the performance bottleneck found in an isolated disk. In this study, we analyze some parallel and distributed file systems, detailing the most interesting and important ones. Finally, we show the use of a parallel file system can be more efficient than the use of a usual local file system, for just one client. acesso concorrente afs bridge ceft-pvfs coda computação distribuída computação paralela disco magnético disco rígido disponibilidade escalabilidade gargalo no sistema de arquivos gfs ide latência nfs nfsp pio pvfs pvfs2 replicação de arquivos sad scsi segurança serviço de arquivos sistema de arquivos sistema de arquivos distribuídos sistema de arquivos paralelos sprite sva tolerância a falhas afs availability bridge ceft-pvfs coda concurrent access distributed computing distributed file system fail over file replication file service file system file system bottleneck gfs hard disk ide latency magnetic disk nfs nfsp parallel computing parallel file system pio pvfs pvfs2 scalability scsi security sprite sva

Page generated in 0.0823 seconds