Clemson University
Optimizing Compression Efficiency with Adaptive Quantization Bit Depths
Abstract
dc:description.abstractLarge-scale scientific instruments and applications generate massive amounts of data, lead- ing to significant challenges in data transfer and storage for analysis. This constitutes a major bottleneck to workflow efficiency and scientific throughput. Lossy compression offers a solution to these storage challenges in increasingly complex systems and services. Error-bounded lossy compression allows users to limit the error introduced during the compression process according to a user-defined metric and achieves significantly higher compression ratios than lossless compression for floating-point data. However, certain data types and compression configurations hinder the attainment of large compression ratios. To address the need for improved compression ratios in hard-to-compress data, this work introduces adaptive error quantization—a novel approach to lossy compression. Adaptive error quantization further refines the quantization of the error between the true value and the reconstructed value. When the original quantized error is excessively large or can be significantly improved, this method dynamically refines the quantized error by adding additional layers of quantization. This process enhances the accuracy of the reconstructed value while incurring minimal overhead. Adaptive error quantization provides a powerful tool for scientists to fine-tune the error introduced in lossy-compressed scientific datasets while simultaneously improving the compression ratio.
Degree
thesis:*- Name thesis:degree_name
- Master of Science (MS)
- Level thesis:degree_level
- Thesis
- Discipline thesis:degree_discipline
- Electrical and Computer Engineering (Holcomb Dept. of)
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Sisk, Carson
- Contributors dc:contributor
-
- Jon C. Calhoun
- Kuang-Ching Wang
- Christophe Darnault
Subjects
dc:subject × 6Identifiers
dc:identifier.*- Repository record dc:identifier
- https://open.clemson.edu/all_theses/4406