Back to search

Boston University

Dynamically tuning LSM tree based databases

Abstract

dc:description.abstract

Log-Structured Merge (LSM) trees are a popular choice of data structure for key-value database systems due to their high ingestion rate and fast reads. They achieve this by appending new writes and updates sequentially and buffering changes in memory before flushing them to disk in sorted order. The LSM tree behavior can be dynamically altered by a large set of tunable parameters to accommodate a wide range of workloads. While these parameters provide flexibility, identifying the optimal value for these parameters, in order to maximize system performance, is a known hard problem. Offline tuning approaches can provide optimal configurations, however, they require knowledge about the workload a priori and/or evaluating hundreds of configurations, meaning that they lack the flexibility to adapt to evolving conditions. In the online setting, evaluating the performance impact of a particular tuning knob can be an expensive endeavor. To this end, we propose Onix, a tuning framework that focuses on tackling the online setting of the tuning problem, specifically dynamically tuning LSM trees using Bayesian Optimization (BO). BO constructs a probabilistic model to navigate the space of tuning knobs, striking a careful balance between exploring uncharted parameter configurations and exploiting areas already identified as promising. We leverage BO’s efficient convergence to minimize the number of configurations deployed for exploration. Onix integrates Microsoft’s BO-based system tuning framework, MLOS, with Meta’s state-of-the-art LSM tree implementation, RocksDB. As workloads are executed on RocksDB, Onix propagates appropriate information to MLOS, which in turn recommends the correct configuration for the current workload. This process is repeated periodically, thus re-evaluating if a new tuning suggestion (i.e., configuration) can provide better performance. In the best-case scenario, Onix can achieve up to 2× better performance (in terms of average read latency) than the default configuration, while performing at least as well as default tuning in the worst case guaranteeing no performance regression.

Degree

thesis:*
Name thesis:degree_name
Master of Science
Level thesis:degree_level
masters
Discipline thesis:degree_discipline
Computer Science
Grantor
Boston University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sharma, Sakshi
Advisor dc:contributor.advisor
  • Athanassoulis, Manos

Subjects

dc:subject × 2

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-ShareAlike 4.0 International
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/49065

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Sharma, Sakshi. Dynamically tuning LSM tree based databases. masters thesis, Boston University, 2024. https://hdl.handle.net/2144/49065