Institutional Repository of Vilnius University
Heterogeninių dokumentų duomenų apjungimas /
Abstract
dc:descriptionMain purpose of this writing was to analyse already available solutions of combination ofheterogeneous document data and using this knowledge write new prototype module. During thiswork several related works were analysed, which showed that there are various ways of joiningheterogeneous document data: from common object request broker architecture to special searchtools likeIBM DB connectandSphereSearch. During analysis, it was made clear that the bestway of combining heterogeneous data is using parallel hash join algorithms, because it can joinseveral results at the same time and only needs to scan all results once to calculate hashes. Querylanguages likeWHIRLandGraphQLshowed how to query heterogeneous data and how to returnjoined results.There are several algorithms presented in this paper necessary for splitting queries, query plan-ning and combination of heterogeneous results. In addition, prototype is implemented, using JA-VA 8 programming language, MongoDB and PostgreSQL databases. Most of the implementationdetails are presented. It is discussed how algorithms were implemented, what problems were fa-ced and how they were resolved. For example, slow query execution in PostgreSQL database forexcessive usage of IN filter was resolved using VALUES function and SQL join. In addition, it isshown how to configure module, how module creates query after splitting, how it transforms fromdatabase result object to project domain objects using Java reflection. Prototype is able to workwith part of SQL operators and aggregates like: =, <, >, >=, <=, LIMIT, OFFSET. Furthermore,performance tests results of prototype are presented, which show that time overhead is small forquery processing actions performed by the module in contrast with all query execution time. Ofcourse, prototype has still some work to be done. Pagination should be revised and way of pagi-nating by all results, not only by one tables result should be found. Furthermore, more aggregatescould be added in a way that not all data should be fetched to main memory to apply them andmore support for NoSQL databases could be added too.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2019
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Zubavičius, Donatas,
- Contributors dc:contributor
-
- Brilingaitė, Agnė
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD35374220&prefLang=en_US