Vortrag zum 2. Linux-Workshop der Max-Planck-Institute 27./28.02.2003 in Leipzig. Im Mittelpunkt stehen Erfahrungen und Vorhaben beim Einsatz von OpenAFS im Rechenzentrum der TU Chemnitz.

Unterlagen zu einem Tutorium im Rahmen des AFS-Workshops 2003 am DESY Zeuthen. Die Suche von Fehlern in komplexen Systemen setzt immer voraus, dass der korrekte Zustand des Systems bekannt ist. Denn nur auf diese Art und Weise kann man erkennen, dass es sich in einer konkreten Situation um ein Fehlverhalten handelt. In diesem Tutorium werden daher Normal- oder Sollzustände von OpenAFS-Fileservern beschrieben. Es werden einzelne Datenstrukturen dargestellt und am Beispiel der Callback-Verwaltung des Fileservers wird gezeigt, wie diese Datenstrukturen zur Laufzeit des Servers organisiert werden.

Heidelberg, Univ., Bach.-Arb., 2006.

Heidelberg, Univ., Diss., 2008. / Online publiziert: 2009.

Vortrag zum 2. Linux-Workshop der Max-Planck-Institute 27./28.02.2003 in Leipzig. Im Mittelpunkt stehen Erfahrungen und Vorhaben beim Einsatz von OpenAFS im Rechenzentrum der TU Chemnitz.

Unterlagen zu einem Tutorium im Rahmen des AFS-Workshops 2003 am DESY Zeuthen. Die Suche von Fehlern in komplexen Systemen setzt immer voraus, dass der korrekte Zustand des Systems bekannt ist. Denn nur auf diese Art und Weise kann man erkennen, dass es sich in einer konkreten Situation um ein Fehlverhalten handelt. In diesem Tutorium werden daher Normal- oder Sollzustände von OpenAFS-Fileservern beschrieben. Es werden einzelne Datenstrukturen dargestellt und am Beispiel der Callback-Verwaltung des Fileservers wird gezeigt, wie diese Datenstrukturen zur Laufzeit des Servers organisiert werden.

Modern disk-storage systems have to accomplish the requirements of a variety of application classes. Applications that process continuous-media data such as video and audio streams require the storage system to guarantee sustained bandwidths. Interactive applications demand the storage system to ensure bounded response times, posing timing constraints on the execution of individual disk requests. Traditional timesharing applications may require both high throughput or overall short response times. With the described applications being more and more used together in todays computing systems, the disk-storage subsystems have to efficiently combine the different requirements of this application mix. In this thesis, I develop the design of a storage system that comprehensively addresses the various challenges posed by including the support for quality-of-service guarantees in disk-storage systems. The presented storage system provides three main properties. First, the admission control includes the support for statistical guarantees to increase the share of the disk bandwidth that can be utilized by the admission control. Second, the disk-request scheduling clearly separates the enforcement of real-time guarantees from the task to establish the optimal execution order of the requests, and it provides a flexible mechanism to combine the execution of requests with different quality-of-service requirements. Finally, the file system addresses both the needs of the former two elements of the storage system and of the various file types used by the applications by providing a flexible block-allocation policy and customized client interfaces. I show the implementation of the presented designs with the DROPS Disk-Storage System and I provide a detailed evaluation based on this implementation.

Modern disk-storage systems have to accomplish the requirements of a variety of application classes. Applications that process continuous-media data such as video and audio streams require the storage system to guarantee sustained bandwidths. Interactive applications demand the storage system to ensure bounded response times, posing timing constraints on the execution of individual disk requests. Traditional timesharing applications may require both high throughput or overall short response times. With the described applications being more and more used together in todays computing systems, the disk-storage subsystems have to efficiently combine the different requirements of this application mix. In this thesis, I develop the design of a storage system that comprehensively addresses the various challenges posed by including the support for quality-of-service guarantees in disk-storage systems. The presented storage system provides three main properties. First, the admission control includes the support for statistical guarantees to increase the share of the disk bandwidth that can be utilized by the admission control. Second, the disk-request scheduling clearly separates the enforcement of real-time guarantees from the task to establish the optimal execution order of the requests, and it provides a flexible mechanism to combine the execution of requests with different quality-of-service requirements. Finally, the file system addresses both the needs of the former two elements of the storage system and of the various file types used by the applications by providing a flexible block-allocation policy and customized client interfaces. I show the implementation of the presented designs with the DROPS Disk-Storage System and I provide a detailed evaluation based on this implementation.

Unterlagen zu einem Workshop im Rahmen des 4. Chemnitzer Linux Tages. Es wird die Installation von AFS-Servern und -Klienten demonstriert. Dabei werden wichtige Eigenschaften gezeigt und einige typische Administrations-Aufgaben erläutert.

This thesis presents a contribution to the field of performance analysis for Input/Output (I/O) related problems, focusing on the area of High Performance Computing (HPC). Beside the compute nodes, High Performance Computing systems need a large amount of supporting components that add their individual behavior to the overall performance characteristic of the whole system. Especially file systems in such environments have their own infrastructure. File operations are typically initiated at the compute nodes and proceed through a deep software stack until the file content arrives at the physical medium. There is a handful of shortcomings that characterize the current state of the art for performance analyses in this area. This includes a system wide data collection, a comprehensive analysis approach for all collected data, an adjusted trace event analysis for I/O related problems, and methods to compare current with archived performance data. This thesis proposes to instrument all soft- and hardware layers to enhance the performance analysis for file operations. The additional information can be used to investigate performance characteristics of parallel file systems. To perform I/O analyses on HPC systems, a comprehensive approach is needed to gather related performance events, examine the collected data and, if necessary, to replay relevant parts on different systems. One larger part of this thesis is dedicated to algorithms that reduce the amount of information that are found in trace files to the level that is needed for an I/O analysis. This reduction is based on the assumption that for this type of analysis all I/O events, but only a subset of all synchronization events of a parallel program trace have to be considered. To extract an I/O pattern from an event trace, only these synchronization points are needed that describe dependencies among different I/O requests. Two algorithms are developed to remove negligible events from the event trace. Considering the related work for the analysis of a parallel file systems, the inclusion of counter data from external sources, e.g. the infrastructure of a parallel file system, has been identified as a major milestone towards a holistic analysis approach. This infrastructure contains a large amount of valuable information that are essential to describe performance effects observed in applications. This thesis presents an approach to collect and subsequently process and store the data. Certain ways how to correctly merge the collected values with application traces are discussed. Here, a revised definition of the term "performance counter" is the first step followed by a tree based approach to combine raw values into secondary values. A visualization approach for I/O patterns closes another gap in the analysis process. Replaying I/O related performance events or event patterns can be done by a flexible I/O benchmark. The constraints for the development of such a benchmark are identified as well as the overall architecture for a prototype implementation. Finally, different examples demonstrate the usage of the developed methods and show their potential. All examples are real use cases and are situated on the HRSK research complex and the 100GBit Testbed at TU Dresden. The I/O related parts of a Bioinformatics and a CFD application have been analyzed in depth and enhancements for both are proposed. An instance of a Lustre file system was deployed and tuned on the 100GBit Testbed by the extensive use of external performance counters.