Back to search

National Technical University of Athens (NTUA)

Διαχείριση και εξερεύνηση μεγάλων συνόλων διασυνδεδεμένων δεδομένων

Abstract

dc:description

This document presents the research contribution regarding the exploration and visualization of very large linked datasets. First, the technologies and innovations that led to the increase of the available big data are discussed. Then the challenges that people interested in the exploration and analysis of the available information are discussed. Emphasis is given in differentiating the challenges related to the nature and characteristics of the available dataset from the ones coming from specific use cases and target audience. Specific, real-world examples are presented to show the needs of the users and the specification for the solutions. Next, a solution that supports users with querying SPARQL endpoints, visualizing the results, in the optimal way based on a knowledge base and a decision support system, and facilitating the exploration of the information through an innovative functionality toolkit is presented. The solution is proposing a client-server architectural model, that allows the users to perform SPARQL queries over any available endpoint, receive the results visualized based on the specific characteristics of the query and explore the visualized information through multiple abstraction and filtering criteria. In addition, a fully-fledged innovative system that supports the representation of any RDF dataset as one continuous graph at the two-dimensional space. The system has been carefully designed to manage any dataset independently of its specific characteristics. The system stores the information in a distributed key-value storage system and indexes the information with a XZ-index ensuring the smooth and timely provision of the information to multiple users regardless the spatial criteria used or the area requested. A dedicated user interface, allows the user to access the information, explore the complete graph, visualize the dataset thought multiple abstraction and filtering criteria, navigate paths of interest or isolate parts of the dataset that wants to further explore.Understanding that the value of the available dataset is closely related to their quality, a technique to improve the quality of the available conversational datasets is proposed. The technique builds on top of semantic relationships, such as synonyms and hyponomy, to calculate the semantic similarity and the semantic relatedness between the topic that the dataset is to be used for and the available information. Taking into consideration the use case that the output dataset is going to be used for, its thematic relation with the source of the input dataset and the language formality needed for the task, the two scores are merged using a weight-based score function into a matching percentage. The dataset is then ranked based on this percentage and only the information above the required threshold is present in the output file. Extended experimental analysis showed that machine learning solutions perform better when trained with smaller but properly created dataset than when trained over complete initial dataset.Finally, the data quality control needed when collecting big datasets is discussed. The specific example of the data collected within the context of the SCENT EU founded project is presented. There volunteers were tasked to use mobile applications and smart sensor to collect images, video and sensor measurement at area of hydrological interest. The collected data were processed in order to collect information about the land cover of the area, the water level and the water velocity of the water body as well as air temperature and soil moisture values. The data were collected from volunteers with no training regarding the proper way to collect scientific measurements, in conditions that were challenging regarding the weather phenomena and the accessibility and in areas that had many technological challenges such as the lack of accurate GPS signal. The collected data are to be used in order to update hydrological models, meaning that there is a need for high accuracy in the measurements used. Innovative techniques that filter out invalid measurements were developed in order to provide the proper data for the models. The techniques were proven to work properly and they were able to support the creation of improved, more accurate flood models.

Degree

thesis:*
Grantor dc:publisher
National Technical University of Athens (NTUA)
Year dc:date
2021

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Krommyda, Maria
  • Κρομμύδα, Μαρία

Subjects

dc:subject × 9

Rights

Language dc:language
gre

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/49987

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Krommyda, Maria; Κρομμύδα, Μαρία. Διαχείριση και εξερεύνηση μεγάλων συνόλων διασυνδεδεμένων δεδομένων. National Technical University of Athens (NTUA), 2021. http://hdl.handle.net/10442/hedi/49987