A large number of personal cloud storage systems have emerged in recent years, such as Dropbox, iCloud, Google Drive etc. A common limitation of these system is that the users have to trust the cloud provider not to be malicious. Now we have a Unity block store, which can solve the problem and provide a secure and durable cloud-based block store. However, the base Unity system does not have the concept of file on top of its block device, thus the concurrent operations to different files can cause false sharing problem. In this thesis, we propose UnityFS, a file system built on top of the base Unity system. We design and implement the file system that maintains a mapping between files and a group of data blocks, such that the whole Unity system can support concurrent file operations to different files from multiple user devices in the personal cloud.

A large number of personal cloud storage systems have emerged in recent years, such as Dropbox, iCloud, Google Drive etc. A common limitation of these system is that the users have to trust the cloud provider not to be malicious. Now we have a Unity block store, which can solve the problem and provide a secure and durable cloud-based block store. However, the base Unity system does not have the concept of file on top of its block device, thus the concurrent operations to different files can cause false sharing problem. In this thesis, we propose UnityFS, a file system built on top of the base Unity system. We design and implement the file system that maintains a mapping between files and a group of data blocks, such that the whole Unity system can support concurrent file operations to different files from multiple user devices in the personal cloud.

This thesis introduces Relative links or rlinks, which are directed labeled links from one file to another in a file system. Rlinks provide a clean way to build and share related-file information without creating additional files and directories. Rlinks form overlay graphs between files of a file system, thus providing useful alternate views of the file system. This thesis implements rlinks for the Linux kernel and modifies the storage structure of the Ext2 file system to store the rlinks.

Storage devices based on Multi-Level Cell (MLC) NAND flash can be found in almost all computer systems except rugged, industrial systems; even though MLC is less expensive and more dense than devices based on standard Single-Level Cell (SLC) NAND flash, MLC’s lower write endurance and lower retention has led system designers to avoid using it. This avoidance is unnecessary in many applications which will never come close to the endurance limits. Furthermore, new processes are leading to storage devices with higher write endurance. System designers should review the specific use-model for their systems and can select MLC-based storage devices when warranted. The result is lower system costs without worry of data loss due to write endurance.

With the advent of emerging "e-Science" applications, today's scientific research increasingly relies on petascale-and-beyond computing over large data sets of the same magnitude. While the computational power of supercomputers has recently entered the era of petascale, the performance of their storage system is far lagged behind by many orders of magnitude. This places an imperative demand on revolutionizing their underlying I/O systems, on which the management of both metadata and data is deemed to have significant performance implications. Prefetching/caching and data locality awareness optimizations, as conventional and effective management techniques for metadata and data I/O performance enhancement, still play their crucial roles in current parallel and distributed file systems. In this study, we examine the limitations of existing prefetching/caching techniques and explore the untapped potentials of data locality optimization techniques in the new era of petascale computing. For metadata I/O access, we propose a novel weighted-graph-based prefetching technique, built on both direct and indirect successor relationship, to reap performance benefit from prefetching specifically for clustered metadata serversan arrangement envisioned necessary for petabyte scale distributed storage systems. For data I/O access, we design and implement Segment-structured On-disk data Grouping and Prefetching (SOGP), a combined prefetching and data placement technique to boost the local data read performance for parallel file systems, especially for those applications with partially overlapped access patterns. One high-performance local I/O software package in SOGP work for Parallel Virtual File System in the number of about 2000 C lines was released to Argonne National Laboratory in 2007 for potential integration into the production mode.

No description available.

Understanding user and system behavior is most vital for designing efficient systems. Most systems are designed with certain user workload in mind. However, such workloads evolve over time, or the underlying hardware assumptions change. Further, most modern systems are not built or deployed in isolation, they interact with other systems whose behavior might not be exactly understood. Thus in order to understand the performance of a system, it must be inspected closely while user workloads are running. Such close inspection must be done with minimum disturbance to the user workload. Thus tracing or collection of all the user and system generated events becomes an important approach in gaining comprehensive insight in user behavior. As part of this work, we have three major contributions. We designed and implemented an in-depth block level I/O tracer, which would collect block level information like sector number, size of the I/O, actual contents of the I/O, along with certain file system information like filename, and offset in the file, for every I/O request. Next, to minimize the impact of the tracing to the running workload, we introduce and implement a sampling mechanism which traces fewer I/O requests. We validate that this sampling preserves certain I/O access patterns. Finally, as one of the application of our tracer, we use it as a crucial component of a system designed to do VM placements according to user workload. / Master of Science

The traditional block-based interface used in flash-based Solid State Drives (SSDs) imposes limitations on performance and endurance due to write amplification and garbage collection overheads. In response to these challenges, the NVMe Zoned Namespaces (ZNS) devices introduces a novel storage interface organized into zones, optimizing garbage collection and reducing write amplification. This research delves into the exploration and profiling of ZNS device characteristics, aiming to enhance user comprehension and utilization. Additionally, the study investigates the integration of ZNS devices into disaggregated storage frameworks to improve resource utilization, proposing server-side management features to simplify client operations and minimize overhead. By offering insights for future development and optimization of ZNS-based storage solutions, this work contributes to advancing storage technology and addressing the shortcomings of traditional block-based interfaces. Through extensive experimentation and analysis, this study sheds light on the optimal configurations and deployment strategies for ZNS-based storage solutions. / Master of Science / Traditional storage drives, like those found in computers and data centers, face challenges that limit their performance and durability. These challenges stem from the way data is stored and managed within these drives, resulting in inefficiencies known as write amplification and garbage collection overheads. To address these issues, a new type of storage device called NVMe Zoned Namespaces (ZNS) has been developed. ZNS devices organize data in a smarter way, grouping it into specific areas called zones. This organization helps to reduce inefficiencies and improve performance. This research explores the characteristics of ZNS devices and how they can be used more effectively. By better understanding and using these devices, we can improve the way data is stored and accessed, leading to faster and more reliable storage solutions. Additionally, this research looks at how ZNS devices can be integrated into larger storage systems to make better use of available resources. Ultimately, this work contributes to advancing storage technology and overcoming the limitations of traditional storage interfaces. We aim to uncover the best ways to deploy and optimize ZNS-based storage solutions for a variety of applications.

Performance-hungry data center applications demand increasingly higher performance from their storage in addition to larger capacity memory at lower cost. While the existing storage technologies (e.g., HDD and flash-based SSD) are limited in their performance, the most prevalent memory technology (DRAM) is unable to address the capacity and cost requirements of these applications. Emerging byte-addressable, non-volatile memory technologies (such as PCM and RRAM) offer performance within an order of magnitude of DRAM, prompting their inclusion in the processor memory subsystem. Such load/store accessible non-volatile or persistent memory (referred to as NVM or PM) introduces an interesting new tier that bridges the performance gap between DRAM and PM, and serves the role of fast storage or slower memory. However, PM has several implications on system design, both hardware and software: (i) the hardware caching mechanisms, while necessary for acceptable performance, complicate the ordering and durability of stores to PM, (ii) the high performance of PM (compared to NAND) and the fact that it is byte-addressable necessitate rethinking of the system software to manage PM and the interfaces to expose PM to the applications, and (iii) the future memory-based applications that will likely employ systems coupling PM with DRAM (for cost and capacity reasons) must be extremely conscious of the performance characteristics of PM and the challenges of using fast vs. slow memory in ways that best meet their performance demands. The key contribution of our research is a set of technologies that addresses these challenges in a bottom-up fashion. Since the real hardware is not yet available, we first implement a hardware emulator that can faithfully emulate the relative performance characteristics of DRAM and PM in a system with separate DRAM and emulated PM regions. We use this emulator to perform all of our evaluations. Next we explore system software support to enable low-overhead PM access by new and legacy applications. Towards this end, we implement PMFS, an optimized light-weight POSIX file system that exploits PM's byte-addressability to avoid overheads of block-oriented storage and enable direct PM access by applications (with memory-mapped I/O). To provide strong consistency guarantees, PMFS requires only a simple hardware primitive that provides software enforceable guarantees of durability and ordering of stores to PM. We demonstrate that PMFS achieves significant (up to an order of magnitude) gains over traditional file systems (such as ext4) on a RAMDISK-like PM block device. Finally, we address the problem of designing memory-based applications for systems with both DRAM and PM by extending our system software to manage both the tiers. We demonstrate for several representative large in-memory applications that it is possible to use a small amount of fast DRAM and large amounts of slower PM without a proportional impact to an application's performance, provided the placement of data structures is done in a careful fashion. To simplify the application programming, we implement a set of libraries and automatic tools (called X-Mem) that enables programmers to achieve optimal data placement with minimal effort on their part. Finally, we demonstrate the potentially large benefits of application-driven memory tiering with X-Mem across a range of applications.

Across the country members of the workforce are being laid off due to downsizing. Most of those people work for large corporations and have access to important company documents. There have been several studies suggesting that employees are taking critical information after learning they will be laid off. This becomes an issue and a threat to a corporation's security. Corporations are then placed in a position to make sure sensitive documents never leave the company. In this study we build a system that is used to assist corporations and systems administrators. This system will prevent users from taking sensitive documents. The system used in this study helps to maintain a level of security that is not only beneficial but is a crucial part of managing a corporation, and enhancing its ability to compete in an aggressive market.