Institutional Repository of Vilnius University
Išmaniųjų apskaitų didelių duomenų kiekių apdorojimas modernioje duomenų apdorojimo architektūroje /
Abstract
dc:descriptionThis work consists of theoretical and experimental parts. The theoretical part briefly introduces the concept of smart meters and their main purpose. It describes the problem of received big data ingestion and processing, presents the relevance of the problem and business needs, introduces the big data life cycle and existing data processing paradigms, evaluates described problem. After evaluation, the real-time processing paradigm was chosen and two streaming processing frameworks „Apache Storm“ and „Apache Spark“ were compared. The conclusion of the comparison is that „Apache Spark“ is best suited to deal with smart meters data handling problem. In the next part, the optimal smart meter data transaction format was defined, approximate expected size was measured and possible data delivery ways were described. Based on the selected stream processing architecture the one of the possible system implementation model for smart meters data ingestion and processing has been suggested. The experimental part describes how the proof of concept was accomplished. The data ingestion and processing simulation empirically tested the system suitability for this task. The obtained results were compared and conclusions were made. In this way, it was shown that the proposed architecture could be implemented and Apache Spark could be used to deal with smart meters data ingestion and processing problem.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2017
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Neviera, Mantas,
- Contributors dc:contributor
-
- Adamonis, Andrius
Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- lit
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD23166011&prefLang=en_US