Harokopio University
A study on the query response time minimization in distributed (spatiotemporal) data stores
Abstract
dc:descriptionData centers have become a critical component in enterprise systems as web services and cloud computing continue their massive growth. Meeting hard application constraints (e.g., temporal) under these circumstances puts an enormous strain on the capabilities and limits of enterprise systems, especially in coping with the immense rate of queries in the persistence layer. To cope with these ever-growing application requirements, the data management layer must decouple itself from the many, heavy disk IOs and add a faster persistence layer in between. This approach is implemented with in-memory storage mechanisms which maintain data views in the form of key-value pairs and respond to queries in near-real time even under heavy loads.However, IMKVS suffer inherently (as any distributed system) from the problem of the uneven load distribution among the underlying nodes. The high heterogeneity combining with the distribution of objects through random identifiers, and the fact that these systems do not take into consideration uneven request workloads (e.g., existence of nodes that hold popular objects), can lead to imbalances. As a consequence, overloaded nodes become hotspots in the system and this has a detrimental effect in the systems’ performance in terms of response time.During my Ph.D., the overarching objective of this thesis was to improve the performance of distributed data stores in terms of queries' execution response times by investigating three different perspectives: performance evaluation of spatiotemporal datastores, effective dynamic load balancing and data compression. Initially, we set to investigate the performance of distributed storage systems. Thus, we investigated the field of spatiotemporal data management and analysis, which is a growing area of research supporting multiple application domains. Part of this thesis was set to identify the most efficient data store system in terms of response times, comparing two of the most representative of the two categories (NoSQL and relational), MongoDB and PostgreSQL for a particular class of requirements, i.e., those that the response time in complex spatiotemporal queries is of high importance. The evaluation is based upon real business scenarios.The performance of distributed data stores can greatly be improved by a more effective object placement. Thus, regarding dynamic load balancing, our proposal is to define the actual contributing factors to the load balancing problem and to provide a novel object placement protocol for distributed storage systems that migrates objects between nodes in run time with the goal to minimize very high (based on a threshold identified empirically) response times. The proposed placement protocol constitutes a hybrid solution which combines consistent hashing and directory services. In addition, we approached the problem of data placement, in terms of data compression. Data compression involves the development of a compact representation of information. Therefore, as part of this Ph.D. dissertation, a number of compression algorithms are proposed which are able to reduce the size of the data, while minimizing the loss and at the same time preserving the quality of the information. In other words, compression is investigated as a means to improve performance.The results showed that the proposed object placement protocol achieves to balance the load on a Redis Cluster and the average response time is also becomes uniform across the nodes, leading to an improved performance. In addition, data compression was presented as an alternative to the load balancing problem. A number of compression algorithms are proposed which are able to reduce the size of the data, while minimizing the loss and at the same time preserving the quality of the information. The relationship and the impact of trajectory compression were investigated experimentally by employing trajectory classification and similarity measure techniques. Finally, regarding the most efficient spatiotemporal distributed storage system, the results demonstrated the he superiority of PostgreSQL in almost all cases with the exception of the polygon intersection queries.
Degree
thesis:*- Grantor dc:publisher
- Harokopio University
- Year dc:date
- 2022
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Makris, Antonios
- Μακρής, Αντώνιος
Subjects
dc:subject × 17- Κατανεμημένα συστήματα λογισμικού
- Καταμερισμός φόρτου
- Κατανεμημένα συστήματα αποθήκευσης
- Χωροχρονικά συστήματα δεδομένων
- Συμπίεση δεδομένων
- NoSQL
- Distributed storage systems
- Load balancing
- Spatiotemporal datastores
- Data compression
- NoSQL databases
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Αυτοματισμοί και Συστήματα ελέγχου
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Automation and control systems
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/51982