{"id":{"repo_id":"mtu","oai_identifier":"oai:digitalcommons.mtu.edu:etdr-2915"},"canonical_url":"https://search.ndltd.org/etd/mtu/oai:digitalcommons.mtu.edu:etdr-2915","repository":{"repo_id":"mtu","name":"Michigan Technological University","base_url":"https://digitalcommons.mtu.edu/do/oai/"},"display":{"title":"Model Guided Memory Optimization For Key-Value Caches","abstract":"Modern web services deploy key-value caches to store popular requests to backend systems. As such, how the cache stores data impacts both the cache miss ratio and throughput. Therefore, in this thesis, we introduce and apply cache modeling techniques to optimize the memory organization of a key-value cache to improve overall cache performance. Specifically, we begin with a single-level key-value cache and use miss ratio curves to adjust the memory assigned to the residing applications dynamically. This leads to an improvement in miss ratio up to 25% over state-of-the-art techniques and an 8.8% improvement in cache throughput. We then consider the implications of memory reassignment for large cache deployments and introduce a faster mechanism that allows the cache to reach steady-state 53% faster. Our patch has been incorporated into the commonly used memcached system. As key-value cache workloads grow in size, we introduce equations to model the behavior of multi- level caches equipped with DRAM and NVM memory for additional capacity. Given the performance disparity between typical DRAM and NVM mediums, we focus on the number of writebacks incurred from L1 to L2. We demonstrate that our closed-form model and trace-based model achieve a 3% error from a simulated two-level cache. Finally, we introduce a novel mechanism for memory organization in a two-level key-value cache that considers whether data should be cached inclusively or exclusively with respect to the second level. Our algorithm, CLUsivity, partitions each class of objects in the cache as inclusive or exclusive and assigns the corresponding amount of DRAM and NVM memory that maximizes performance. Our results show that CLUsivity can improve overall cache throughput relative to an inclusive cache while maintaining a miss ratio comparable to an exclusive cache.","abstract_html":"Modern web services deploy key-value caches to store popular requests to backend systems. As such, how the cache stores data impacts both the cache miss ratio and throughput. Therefore, in this thesis, we introduce and apply cache modeling techniques to optimize the memory organization of a key-value cache to improve overall cache performance. Specifically, we begin with a single-level key-value cache and use miss ratio curves to adjust the memory assigned to the residing applications dynamically. This leads to an improvement in miss ratio up to 25% over state-of-the-art techniques and an 8.8% improvement in cache throughput. We then consider the implications of memory reassignment for large cache deployments and introduce a faster mechanism that allows the cache to reach steady-state 53% faster. Our patch has been incorporated into the commonly used memcached system. As key-value cache workloads grow in size, we introduce equations to model the behavior of multi- level caches equipped with DRAM and NVM memory for additional capacity. Given the performance disparity between typical DRAM and NVM mediums, we focus on the number of writebacks incurred from L1 to L2. We demonstrate that our closed-form model and trace-based model achieve a 3% error from a simulated two-level cache. Finally, we introduce a novel mechanism for memory organization in a two-level key-value cache that considers whether data should be cached inclusively or exclusively with respect to the second level. Our algorithm, CLUsivity, partitions each class of objects in the cache as inclusive or exclusive and assigns the corresponding amount of DRAM and NVM memory that maximizes performance. Our results show that CLUsivity can improve overall cache throughput relative to an inclusive cache while maintaining a miss ratio comparable to an exclusive cache.","abstract_has_math":false,"creators":["Byrne, Daniel"],"institution":null,"degree_name":"Doctor of Philosophy in Computer Science (PhD)","degree_level":"Open Access Dissertation","degree_discipline":"Department of Computer Science","degree_department":null,"school":null,"contributors":["Zhenlin Wang","Nilufer Onder"],"advisors":[],"committee_chairs":[],"committee_members":["Chen Ding","Soner Onder"],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:19:12Z","subjects":["Key Value Caches","Memory Management","Cache Modeling","Multi-level Caching","Databases and Information Systems","Systems Architecture","Theory and Algorithms"],"languages":[],"rights":[],"rights_urls":["http://creativecommons.org/licenses/by-nc/4.0/"],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.mtu.edu/etdr/1800"],"render_values":[{"text":"https://digitalcommons.mtu.edu/etdr/1800","href":"https://digitalcommons.mtu.edu/etdr/1800","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.37099/mtu.dc.etdr/1800","outbound_label":"DOI","outbound_source":"dc:identifier","record_url":"https://doi.org/10.37099/mtu.dc.etdr/1800","record_label":"Repository record","record_kind":"doi","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.mtu.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.mtu.edu%3Aetdr-2915","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zhenlin Wang","Nilufer Onder"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Chen Ding","Soner Onder"]},{"key":"dc:creator","label":"Author","values":["Byrne, Daniel"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.created","label":"Dc Date Created","values":["2024-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Department of Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Open Access Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy in Computer Science (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Key Value Caches","Memory Management","Cache Modeling","Multi-level Caching","Databases and Information Systems","Systems Architecture","Theory and Algorithms"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights.license","label":"Dc Rights License","values":["http://creativecommons.org/licenses/by-nc/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["10.37099/mtu.dc.etdr/1800","https://digitalcommons.mtu.edu/etdr/1800"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Modern web services deploy key-value caches to store popular requests to backend systems. As such, how the cache stores data impacts both the cache miss ratio and throughput. Therefore, in this thesis, we introduce and apply cache modeling techniques to optimize the memory organization of a key-value cache to improve overall cache performance. Specifically, we begin with a single-level key-value cache and use miss ratio curves to adjust the memory assigned to the residing applications dynamically. This leads to an improvement in miss ratio up to 25% over state-of-the-art techniques and an 8.8% improvement in cache throughput. We then consider the implications of memory reassignment for large cache deployments and introduce a faster mechanism that allows the cache to reach steady-state 53% faster. Our patch has been incorporated into the commonly used memcached system. As key-value cache workloads grow in size, we introduce equations to model the behavior of multi- level caches equipped with DRAM and NVM memory for additional capacity. Given the performance disparity between typical DRAM and NVM mediums, we focus on the number of writebacks incurred from L1 to L2. We demonstrate that our closed-form model and trace-based model achieve a 3% error from a simulated two-level cache. Finally, we introduce a novel mechanism for memory organization in a two-level key-value cache that considers whether data should be cached inclusively or exclusively with respect to the second level. Our algorithm, CLUsivity, partitions each class of objects in the cache as inclusive or exclusive and assigns the corresponding amount of DRAM and NVM memory that maximizes performance. Our results show that CLUsivity can improve overall cache throughput relative to an inclusive cache while maintaining a miss ratio comparable to an exclusive cache."]},{"key":"dc:title","label":"Title","values":["Model Guided Memory Optimization For Key-Value Caches"]}]}],"canonical_facts":{"dc:contributor":["Zhenlin Wang","Nilufer Onder"],"dc:contributor.committeemember":["Chen Ding","Soner Onder"],"dc:creator":["Byrne, Daniel"],"dc:date.created":["2024-01-01T08:00:00Z"],"dc:description.abstract":["Modern web services deploy key-value caches to store popular requests to backend systems. As such, how the cache stores data impacts both the cache miss ratio and throughput. Therefore, in this thesis, we introduce and apply cache modeling techniques to optimize the memory organization of a key-value cache to improve overall cache performance. Specifically, we begin with a single-level key-value cache and use miss ratio curves to adjust the memory assigned to the residing applications dynamically. This leads to an improvement in miss ratio up to 25% over state-of-the-art techniques and an 8.8% improvement in cache throughput. We then consider the implications of memory reassignment for large cache deployments and introduce a faster mechanism that allows the cache to reach steady-state 53% faster. Our patch has been incorporated into the commonly used memcached system. As key-value cache workloads grow in size, we introduce equations to model the behavior of multi- level caches equipped with DRAM and NVM memory for additional capacity. Given the performance disparity between typical DRAM and NVM mediums, we focus on the number of writebacks incurred from L1 to L2. We demonstrate that our closed-form model and trace-based model achieve a 3% error from a simulated two-level cache. Finally, we introduce a novel mechanism for memory organization in a two-level key-value cache that considers whether data should be cached inclusively or exclusively with respect to the second level. Our algorithm, CLUsivity, partitions each class of objects in the cache as inclusive or exclusive and assigns the corresponding amount of DRAM and NVM memory that maximizes performance. Our results show that CLUsivity can improve overall cache throughput relative to an inclusive cache while maintaining a miss ratio comparable to an exclusive cache."],"dc:identifier":["10.37099/mtu.dc.etdr/1800","https://digitalcommons.mtu.edu/etdr/1800"],"dc:rights.license":["http://creativecommons.org/licenses/by-nc/4.0/"],"dc:subject":["Key Value Caches","Memory Management","Cache Modeling","Multi-level Caching","Databases and Information Systems","Systems Architecture","Theory and Algorithms"],"dc:title":["Model Guided Memory Optimization For Key-Value Caches"],"thesis:degree_discipline":["Department of Computer Science"],"thesis:degree_level":["Open Access Dissertation"],"thesis:degree_name":["Doctor of Philosophy in Computer Science (PhD)"]},"updated_at":"2026-11-19T08:19:12Z"}