Back to search

Athens University Economics and Business (AUEB)

Block-level storage in structured overlay networks: the design and implementation of a distributed hash table-based storage system

Abstract

dc:description

The ideal storage system is globally accessible, always available, provides unlimited performance and capacity for an unlimited number of clients, requires no management and can be applied to any storage environment. Over the last twenty years great effort has been made by the research community to design and evaluate storage systems that approximate the ideal one. Notable innovations include hardware-based storage systems, distributed block-based storage systems, remote and distributed file systems. The storage systems deployed so far are tailored for specific storage environments. More specifically, hardware-based storage systems are very expensive in terms of deployment and administration costs, while they do not scale well as the demand for more storage space and storage servers grows. Distributed file systems use dedicated metadata servers that accessed during every file operation resulting to possible deficiencies and bottlenecks. Moreover, they cannot support high performance enterprise applications which access the underlying storage infrastructure directly bypassing the caching mechanism provided by the operating system. The client-server block devices proposed so far suffer from scalability issues, since a single server keeps all the available storage space. On the other hand, distributed block-based storage systems overcome this problem albeit they utilize global data structured that must be maintained across all storage nodes. This fact results to limited scalability because the basic parameters of these systems must be statically assigned when the system is initiated. Different storage environments present great differentiation, in terms of system requirements and constraints that must be taken into account when designing a storage infrastructure. For instance, enterprise storage systems should support high I/O throughput, system resilience even after catastrophic failures, data reliability as well as low deployment, administration and maintenance costs. On the other hand, I/O performance would not be the first priority of a wide-area cooperative storage infrastructure, since performance in such environments is bounded by the maximum available bandwidth of each client. The proposed approach is based on the combination of block-based storage clients, which are implemented as low-level kernel modules, with the special properties that peer-to-peer overlay networks can provide. Following this approach, we argue that an overlay network, which acts as a generic storage infrastructure available to an arbitrary number of clients, can be deployed in every target storage environment. Minimum tuning of the overlay network parameters allows the direct adoption of such an approach in these environments, while the desirable system properties are maintained in acceptable levels. Our approach has been evaluated in a computer cluster and in an emulated wide-area wireless network. The main contributions of this thesis are: · A thorough analysis of the design requirements that a distributed storage system should satisfy so that it can provide an efficient solution when deployed in high performance clusters and large scale networks. · The design and the open source implementation of such a system. · An extensive evaluation of the proposed system. · Regarding wide area cooperative environments, like a wireless community network, we have also contributed an implementation of an open source system for measuring QoS indicators. This system complements the storage system. The experimental evaluation of our system indicates its viability in both cluster and wide-area storage environments. More specifically, in a cluster storage environment we show that the proposed system can perform equally to well-known distributed storage systems, while it behaves exceptionally for several access patterns, like direct I/O. In wide-area environments we prove that our system can saturate clients’ links for both read and write I/O patterns even in high latency network infrastructures, due to its ability to parallelize, when possible, the I/O procedure.

Degree

thesis:*
Grantor dc:publisher
Athens University Economics and Business (AUEB)
Year dc:date
2009

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Parisis, Georgios-Aggelos
  • Παρίσης, Γεώργιος-Άγγελος

Subjects

dc:subject × 17

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/17823

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Parisis, Georgios-Aggelos; Παρίσης, Γεώργιος-Άγγελος. Block-level storage in structured overlay networks: the design and implementation of a distributed hash table-based storage system. Athens University Economics and Business (AUEB), 2009. http://hdl.handle.net/10442/hedi/17823