No description available.

Database systems have long been optimized to hide the higher latency of storage media, yielding complex persistence mechanisms. With the advent of large DRAM capacities, it became possible to keep a full copy of the data in DRAM. Systems that leverage this possibility, such as main-memory databases, keep two copies of the data in two different formats: one in main memory and the other one in storage. The two copies are kept synchronized using snapshotting and logging. This main-memory-centric architecture yields nearly two orders of magnitude faster analytical processing than traditional, disk-centric ones. The rise of Big Data emphasized the importance of such systems with an ever-increasing need for more main memory. However, DRAM is hitting its scalability limits: It is intrinsically hard to further increase its density. Storage-Class Memory (SCM) is a group of novel memory technologies that promise to alleviate DRAM’s scalability limits. They combine the non-volatility, density, and economic characteristics of storage media with the byte-addressability and a latency close to that of DRAM. Therefore, SCM can serve as persistent main memory, thereby bridging the gap between main memory and storage. In this dissertation, we explore the impact of SCM as persistent main memory on database systems. Assuming a hybrid SCM-DRAM hardware architecture, we propose a novel software architecture for database systems that places primary data in SCM and directly operates on it, eliminating the need for explicit IO. This architecture yields many benefits: First, it obviates the need to reload data from storage to main memory during recovery, as data is discovered and accessed directly in SCM. Second, it allows replacing the traditional logging infrastructure by fine-grained, cheap micro-logging at data-structure level. Third, secondary data can be stored in DRAM and reconstructed during recovery. Fourth, system runtime information can be stored in SCM to improve recovery time. Finally, the system may retain and continue in-flight transactions in case of system failures. However, SCM is no panacea as it raises unprecedented programming challenges. Given its byte-addressability and low latency, processors can access, read, modify, and persist data in SCM using load/store instructions at a CPU cache line granularity. The path from CPU registers to SCM is long and mostly volatile, including store buffers and CPU caches, leaving the programmer with little control over when data is persisted. Therefore, there is a need to enforce the order and durability of SCM writes using persistence primitives, such as cache line flushing instructions. This in turn creates new failure scenarios, such as missing or misplaced persistence primitives. We devise several building blocks to overcome these challenges. First, we identify the programming challenges of SCM and present a sound programming model that solves them. Then, we tackle memory management, as the first required building block to build a database system, by designing a highly scalable SCM allocator, named PAllocator, that fulfills the versatile needs of database systems. Thereafter, we propose the FPTree, a highly scalable hybrid SCM-DRAM persistent B+-Tree that bridges the gap between the performance of transient and persistent B+-Trees. Using these building blocks, we realize our envisioned database architecture in SOFORT, a hybrid SCM-DRAM columnar transactional engine. We propose an SCM-optimized MVCC scheme that eliminates write-ahead logging from the critical path of transactions. Since SCM -resident data is near-instantly available upon recovery, the new recovery bottleneck is rebuilding DRAM-based data. To alleviate this bottleneck, we propose a novel recovery technique that achieves nearly instant responsiveness of the database by accepting queries right after recovering SCM -based data, while rebuilding DRAM -based data in the background. Additionally, SCM brings new failure scenarios that existing testing tools cannot detect. Hence, we propose an online testing framework that is able to automatically simulate power failures and detect missing or misplaced persistence primitives. Finally, our proposed building blocks can serve to build more complex systems, paving the way for future database systems on SCM.

Submitted by Caroline Xavier (caroline.xavier@pucrs.br) on 2017-07-03T15:03:25Z No. of bitstreams: 1 TES_TACIANO_DRECKMANN_PEREZ_COMPLETO.pdf: 1779781 bytes, checksum: 2111ccea963be0eea76bdbb7d24321d9 (MD5) / Made available in DSpace on 2017-07-03T15:03:25Z (GMT). No. of bitstreams: 1 TES_TACIANO_DRECKMANN_PEREZ_COMPLETO.pdf: 1779781 bytes, checksum: 2111ccea963be0eea76bdbb7d24321d9 (MD5) Previous issue date: 2017-05-03 / Sistemas computacionais da atualidade tradicionalmente separam mem?ria e armazenamento. Linguagens de programa??o tipicamente refletem essa distin??o usando diferentes representa??es para dados em mem?ria (ex. estruturas de dados, objetos) e armazenamento (ex. arquivos, bancos de dados). A movimenta??o de dados entre esses dois n?veis e representa??es, bidirecionalmente, compromete tanto a efici?ncia do programador quanto de execu??o dos programas. Tecnologias recentes de memoria n?o-vol?til, tais como mem?ria de transi??o de fase, resistiva e magnetoresistiva, possibilitam combinar mem?ria principal e armazenamento em uma ?nica entidade de mem?ria persistente, abrindo caminho para abstra??es mais eficientes para lidar com persist?ncia de dados. Essa tese de doutorado introduz uma abordagem de projeto para o ambiente de execu??o de linguagens com ger?ncia autom?tica de mem?ria, baseado numa combina??o original de persist?ncia ortogonal, programa??o para mem?ria persistente, persist?ncia por alcance, e transa??es com atomicidade em caso de falha. Esta abordagem pode melhorar significativamente a produtividade do programador e a efici?ncia de execu??o dos programas, uma vez que estruturas de dados em mem?ria passam a ser persistentes de forma transparente, sem a necessidade de programar explicitamente o armazenamento, e removendo a necessidade de cruzar fronteiras sem?nticas. De forma a validar e demonstrar a abordagem proposta, esse trabalho tamb?m apresenta JaphaVM, a primeira M?quina Virtual Java especificamente projetada para mem?ria persistente. Resultados experimentais usando benchmarks e aplica??es reais demonstram que a JaphaVM, na maioria dos casos, executa as mesmas opera??es cerca de uma a duas ordens de magnitude mais rapidamente do que implementa??es equivalentes usando bancos de dados ou arquivos, e, ao mesmo tempo, requer significativamente menos linhas de c?digo. / Current computer systems separate main memory from storage. Programming languages typically reflect this distinction using different representations for data in memory (e.g. data structures, objects) and storage (e.g. files, databases). Moving data back and forth between these different layers and representations compromise both programming and execution efficiency. Recent nonvolatile memory technologies, such as Phase-Change Memory, Resistive RAM, and Magnetoresistive RAM make it possible to collapse main memory and storage into a single layer of persistent memory, opening the way for simpler and more efficient programming abstractions for handling persistence. This Ph.D. thesis introduces a design for the runtime environment for languages with automatic memory management, based on an original combination of orthogonal persistence, persistent memory programming, persistence by reachability, and lock-based failure-atomic transactions. Such design can significantly increase programming and execution efficiency, as in-memory data structures are transparently persistent, without the need for programmatic persistence handling, and removing the need for crossing semantic boundaries. In order to validate and demonstrate the proposed concepts, this work also presents JaphaVM, the first Java Virtual Machine specifically designed for persistent memory. In experimental results using benchmarks and real-world applications, JaphaVM in most cases executed the same operations between one and two orders of magnitude faster than database- and file-based implementations, while requiring significantly less lines of code.