Aristotle University Of Thessaloniki (AUTH)
Ιεραρχικός αλγόριθμος ομαδοποίησης κατηγορικών δεδομένων βάσει συχνότητας για ειδικούς πεδίου εφαρμογής
Abstract
dc:descriptionMost sciences today are in a state of rapid transition towards the new era of Big Data, which is characterized by the collection and processing of large data volumes aimed at extracting new knowledge. Increasing numbers of users are gaining access to the data, exploring and developing new techniques and algorithms for storage, processing, knowledge extraction, and visualization of results. Data Science encompasses models, methods, tools, and techniques that transform data of various types into clear information, which can be handled even by analysts with limited computing skills. Machine Learning plays a central role in Data Science, as it is part of both the analytical and the prediction process. A typical data analysis process usually includes the following stages: data understanding, data preparation/preprocessing, analysis, modeling, evaluation, visualization, and decision-making. Clustering is a Machine Learning technique used for organizing data into homogeneous groups, extracting patterns, and reducing the complexity of the problem. Hierarchical clustering algorithms retain more information about the groups and their interconnections, as they are organized in dendrograms that are useful in various applications, while not requiring initialization of the number of clusters and their centers. The main drawback of hierarchical clustering algorithms is that they rely on the computation of pairwise similarities between the elements of the entire dataset, in order to generate the clusters from scratch. This technique greatly increases the complexity of the problem, both in terms of memory consumption and execution time, which is not easily manageable for large volumes of data. The purpose of the dissertation was to develop a hierarchical clustering algorithm for categorical data based on features, suitable for domain experts. The algorithm should provide clean clusters represented by specific features, while overcoming the difficulties of classical hierarchical algorithms regarding memory consumption and execution time. The presentation of the results to the user should be done through a user-friendly environment that can assist the domain experts in their decision-making process. For this purpose, the FBC hierarchical clustering algorithm was developed, which is based on the frequency of appearance of features and overcomes the aforementioned difficulties of classical hierarchical algorithms. The algorithm is efficient, scalable, user-friendly, and offers the possibility of visualizing the results of the overall clustering using a binary tree or a graph, as well as the content of each group. To evaluate the algorithm and explore its generalization capability, three different usage scenarios were implemented that analyze different types of data. In the first scenario, gene sequence data (proteins) were used, which consist of very long sequences of characters. The second scenario handled text documents, where characters are organized into words, sentences, paragraphs, etc. In the third scenario, the data concerned fashion-clothing characteristics, and the algorithm was integrated into an AI fashion designer assistant system, demonstrating its usefulness and efficiency in practical applications.
Degree
thesis:*- Grantor dc:publisher
- Aristotle University Of Thessaloniki (AUTH)
- Year dc:date
- 2023
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kotouza, Maria
- Κωτούζα, Μαρία
Subjects
dc:subject × 14- Ιεραρχική ομαδοποίηση
- Κατηγορικά δεδομένα
- Ανάλυση δεδομένων
- Οπτικοποίηση αποτελεσμάτων
- Hierarchical clustering
- Nominal data
- Data analysis
- Results visualization
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/56386