Back to search

Clemson University

Lossy Compression and Its Application on Large Scale Scientific Datasets

Abstract

dc:description.abstract

High Performance Computing (HPC) applications are always expanding in data size and computational complexity. It is becoming necessary to consider fault tolerance and system recovery to reduce computation and resource cost in HPC systems. The computation of modern large scale HPC applications are facing bottleneck due to computation complexities, increased runtime and large data storage requirements. These issues can not be ignored in current supercomputing era. Data compression is one of the effective ways to address data storage issue. Among data compression, the lossy compression is much more feasible and efficient than the traditional lossless compression due to low I/O bandwidth of large applications. The goal of this work is to observe and find the optimal lossy compression configuration which has the minimal user controlled error with maximum compression ratio. For this purpose two large scale application have been experimented with various parameters of well known compression method called SZ. The first application is a quantum chemistry based HPC application NWChem. The second application is the vascular blood flow simulation data generated by parallel lattice Boltzmann code for fluid flow simulations with complex geometries called HemeLB. SZ compressor is integrated in the applications' code for testing the correctness and scalability and give a comparative picture of the performance change. Lastly the statistical methods are tested to pre-determine the data distortion for different error bounds.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Electrical and Computer Engineering (Holcomb Dept. of)

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Reza, Tasmia
Contributors dc:contributor
  • Jon Calhoun
  • Walt Ligon
  • Melissa Smith
  • Ulf Schiller

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Repository record dc:identifier
https://open.clemson.edu/all_theses/3698

Chain of custody

source
Harvested from
Clemson University
Base URL
open.clemson.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Reza, Tasmia. Lossy Compression and Its Application on Large Scale Scientific Datasets. Thesis thesis, https://open.clemson.edu/all_theses/3698